←back to thread

AI companies cause most of traffic on forums

(pod.geraspora.de)
770 points ta988 | 1 comments | 30 Dec 24 14:37 UTC | HN request time: 0.305s | source
Show context
johng ◴[30 Dec 24 14:40 UTC] No.42549646[source]▶
If they ignore robots.txt there should be some kind of recourse :(
replies(5): >>42550889 #>>42550903 #>>42550975 #>>42551033 #>>42551045 #
exe34 ◴[30 Dec 24 16:48 UTC] No.42550975[source]▶
zip b*mbs?
replies(1): >>42551098 #
brookst ◴[30 Dec 24 17:00 UTC] No.42551098[source]▶
Assuming there is at least one already linked somewhere on the web, the crawlers already have logic to handle these.
replies(1): >>42551315 #
exe34 ◴[30 Dec 24 17:21 UTC] No.42551315[source]▶
if you can detect them, maybe feed them low iq stuff from a small llama. add latency to waste their time.
replies(1): >>42551638 #
brookst ◴[30 Dec 24 17:49 UTC] No.42551638[source]▶
It would cost you more than it costs them. And there is enough low IQ stuff from humans that they already do tons of data cleaning.
replies(1): >>42552248 #
1. sangnoir ◴[30 Dec 24 18:45 UTC] No.42552248[source]▶
> And there is enough low IQ stuff from humans that they already do tons of data cleaning

Whatever cleaning they do is not effective, simply because it cannot scale with the sheer volumes if data they ingest. I had an LLM authoritatively give an incorrect answer, and when I followed up to the source, it was from a fanfic page.

Everyone ITT who's being told to give up because its hopeless to defend against AI scrapers - you're being propagandized, I won't speculate on why - but clearly this is an arms race with no clear winner yet. Defenders are free to use LLM to generate chaff.