←back to thread

Ban me at the IP level if you don't like me

(boston.conman.org)
597 points classichasclass | 2 comments | 25 Aug 25 04:23 UTC | HN request time: 0.417s | source
Show context
PeterStuer ◴[25 Aug 25 06:30 UTC] No.45010862[source]▶
FAFO from both sides. Not defending this bot at all. That said, the shenanigans some rogue or clueless webmasters are up to blocking legitimate and non intrusive or load causing M2M trafic is driving some projects into the arms of 'scrape services' that use far less considerate nor ethical means to get to the data you pay them for.

IP blocking is useless if your sources are hundreds of thousands of people worldwide just playing a "free" game on their phone that once in a while on wifi fetches some webpages in the background for the game publisher's scraping as a service side revenue deal.

replies(3): >>45011013 #>>45011442 #>>45011662 #
1. ahtihn ◴[25 Aug 25 06:50 UTC] No.45011013[source]▶
What? Are you trying to say it's legitimate to want to scrape websites that are actively blocking you because you think you are "not intrusive"? And that this justifies paying for bad actors to do it for you?

I can't believe the entitlement.

replies(1): >>45011095 #
2. PeterStuer ◴[25 Aug 25 07:03 UTC] No.45011095[source]▶
No. I'm talking about literally legitimate, information that has to be public by law and/or regulation (typically gov stuff), in formats specifically meant for m2m consuption, and still blocked by clueless or malicious outsourced lowest bidder site managers.

And no, I do not use those paid services, even though it would make it much easier.