Nepenthes is a tarpit to catch AI web crawlers

> ANY SITE THIS SOFTWARE IS APPLIED TO WILL LIKELY DISAPPEAR FROM ALL SEARCH RESULTS

Bug, or feature, this? Could be a way to keep your site public yet unfindable.

You can already do this with a robots.txt file

Technically speaking, yes - but it's in no way enforced, as far as I understand it's more of an honour system.

This malicious solution aligns with incentives (or, disincentives) of the parasitic actors, and might be practically more effective.

> If other pages point to your page with descriptive text, Google could still index the URL without visiting the page. If you want to block your page from search results, use another method such as password protection or noindex.

> If your web page is blocked with a robots.txt file, its URL can still appear in search results, but the search result will not have a description.

https://developers.google.com/search/docs/crawling-indexing/...

So, a robots.txt will not keep your site off of google, it just prevents it from getting crawled. (But, to be fair, this tool probably does not do this as well)