←back to thread

Nepenthes is a tarpit to catch AI web crawlers

(zadzmo.org)
646 points blendergeek | 1 comments | 16 Jan 25 13:57 UTC | HN request time: 0s | source
Show context
observationist[dead post] ◴[16 Jan 25 15:30 UTC] No.42726591[source]▶
[flagged]
jsheard ◴[16 Jan 25 15:39 UTC] No.42726726[source]▶
This is a really bad take, it's not like this server is hacking clients which connect to it. It's providing perfectly valid HTTP responses that just happen to be slow and full of markov gibberish, any harm which comes of that is self inflicted by assuming that websites must provide valuable data as a matter of course.

If AI companies want to sue webmasters for that then by all means, they can waste their money and get laughed out of court.

replies(3): >>42726813 #>>42726898 #>>42729375 #
observationist[dead post] ◴[16 Jan 25 15:45 UTC] No.42726813[source]▶
[flagged]
1. tofof ◴[16 Jan 25 17:12 UTC] No.42728003[source]▶
He's not interfering with any normal operation of any system. He is offering links. You can follow them or not, entirely at your own discretion. Those links load slowly. You can wait for them to complete or not, entirely at your own discretion.

The crawler's normal operation is not interfered with in any way: the crawler does exactly what it's programmed to do. If its programmers decided it should exhaustively follow links, he's not preventing it from doing that operation.

Legally, at best you'd be looking to warp the concept of attractive nuisance to apply to a crawler. As that legal concept is generally intended to prevent bodily harm to children, however, good luck.