←back to thread

Cloudflare.com's Robots.txt

(www.cloudflare.com)
145 points sans_souse | 1 comments | 17 Nov 24 12:39 UTC | HN request time: 0s | source
Show context
palsecam ◴[17 Nov 24 14:14 UTC] No.42164269[source]▶
That’s a funny one!

Anyone knows of others like that?

Here is mine: https://FreeSolitaire.win/robots.txt

replies(2): >>42164327 #>>42165802 #
jsheard ◴[17 Nov 24 14:24 UTC] No.42164327[source]▶
Google used to have a /killer-robots.txt which forbid the T-1000 and T-800 from accessing Larry Page and Sergey Brin, but they took that down at some point.
replies(1): >>42165463 #