(github.com)

545 points mmh0000 | 2 comments | 03 Apr 25 15:24 UTC | HN request time: 1.809s | source

1. matt-p ◴[04 Apr 25 00:25 UTC] No.43577054[source]▶

I do kind of yern for the simpler days when if a website didn't mind bots it allowed it and if they did they blocked your user agent.

replies(2): >>43578244 #>>43588115 #

2. andrethegiant ◴[04 Apr 25 04:09 UTC] No.43578244[source]▶

>>43577054 (TP) #

Back then websites weren’t so resource intensive. The negative backlash towards bots is kind of a side effect of how demanding expectations of web experiences has become.

↑

Curl-impersonate: Special build of curl that can impersonate the major browsers