←back to thread

AI scrapers request commented scripts

(cryptography.dog)
257 points ColinWright | 1 comments | 31 Oct 25 15:44 UTC | HN request time: 0s | source
Show context
Noumenon72 ◴[31 Oct 25 17:25 UTC] No.45774469[source]▶
It doesn't seem that abusive. I don't comment things out thinking "this will keep robots from reading this".
replies(2): >>45774493 #>>45774628 #
mostlysimilar ◴[31 Oct 25 17:42 UTC] No.45774628[source]▶
The article mentions using this as a means of detecting bots, not as a complaint that it's abusive.

EDIT: I was chastised, here's the original text of my comment: Did you read the article or just the title? They aren't claiming it's abusive. They're saying it's a viable signal to detect and ban bots.

replies(3): >>45774645 #>>45774743 #>>45776844 #
woodrowbarlow ◴[31 Oct 25 17:51 UTC] No.45774743[source]▶
the first few words of the article are:

> Last Sunday I discovered some abusive bot behaviour [...]

replies(2): >>45774770 #>>45774783 #
1. mostlysimilar ◴[31 Oct 25 17:53 UTC] No.45774770{3}[source]▶
> The robots.txt for the site in question forbids all crawlers, so they were either failing to check the policies expressed in that file, or ignoring them if they had.