←back to thread

Nepenthes is a tarpit to catch AI web crawlers

(zadzmo.org)
646 points blendergeek | 1 comments | 16 Jan 25 13:57 UTC | HN request time: 0.242s | source
Show context
quchen ◴[16 Jan 25 14:32 UTC] No.42725651[source]▶
Unless this concept becomes a mass phenomenon with many implementations, isn’t this pretty easy to filter out? And furthermore, since this antagonizes billion-dollar companies that can spin up teams doing nothing but browse Github and HN for software like this to prevent polluting their datalakes, I wonder whether this is a very efficient approach.
replies(9): >>42725708 #>>42725957 #>>42725983 #>>42726183 #>>42726352 #>>42726426 #>>42727567 #>>42728923 #>>42730108 #
1. reedf1 ◴[16 Jan 25 15:14 UTC] No.42726352[source]▶
The idea is that you place this in parallel to the rest of your website routes, that way your entire server might get blacklisted by the bot.