←back to thread

AI web crawlers are destroying websites in their never-ending content hunger

(www.theregister.com)
211 points CrankyBear | 1 comments | 02 Sep 25 16:24 UTC | HN request time: 0.199s | source
Show context
giancarlostoro ◴[02 Sep 25 17:27 UTC] No.45106227[source]▶
I'm not sure why they don't just cache the websites and avoid going back for at least 24 hours, especially in the case of most sites. I swear its like we're re-learning software engineering basics with LLMs / AI and it kills me.
replies(8): >>45106404 #>>45106430 #>>45106554 #>>45107000 #>>45107104 #>>45107170 #>>45107187 #>>45112971 #
1. gowld ◴[02 Sep 25 18:28 UTC] No.45107104[source]▶
Who says they don't?