←back to thread

AI companies cause most of traffic on forums

(pod.geraspora.de)
770 points ta988 | 1 comments | 30 Dec 24 14:37 UTC | HN request time: 0.243s | source
1. jsheard ◴[30 Dec 24 17:11 UTC] No.42551211[source]▶
It won't help with the more egregious scrapers, but this list is handy for telling the ones that do respect robots.txt to kindly fuck off:

https://github.com/ai-robots-txt/ai.robots.txt