←back to thread

192 points beedeebeedee | 4 comments | | HN request time: 0.606s | source
Show context
kazinator ◴[] No.41908045[source]
OTOH: ByteDance intern responsible for spamming your web server with crawlers that ignore robots.txt given permanent position with a raise, now in management.
replies(1): >>41910007 #
1. 123yawaworht456 ◴[] No.41910007[source]
honoring robots.txt is an informal courtesy, not international law.
replies(1): >>41910655 #
2. davemp ◴[] No.41910655[source]
Not breaking the law is just about the lowest bar you can set for an organization.
replies(2): >>41911135 #>>41914451 #
3. not_a_bot_4sho ◴[] No.41911135[source]
We can go lower
4. hnfong ◴[] No.41914451[source]
FYI, we're still not sure whether the scraped AI training datasets involve copyright infringement.