←back to thread

Cloudflare.com's Robots.txt

(www.cloudflare.com)
145 points sans_souse | 2 comments | 17 Nov 24 12:39 UTC | HN request time: 0.712s | source
Show context
palsecam ◴[17 Nov 24 14:14 UTC] No.42164269[source]▶
That’s a funny one!

Anyone knows of others like that?

Here is mine: https://FreeSolitaire.win/robots.txt

replies(2): >>42164327 #>>42165802 #
1. jsheard ◴[17 Nov 24 14:24 UTC] No.42164327[source]▶
Google used to have a /killer-robots.txt which forbid the T-1000 and T-800 from accessing Larry Page and Sergey Brin, but they took that down at some point.
replies(1): >>42165463 #
2. shaunpud ◴[17 Nov 24 17:29 UTC] No.42165463[source]▶
https://web.archive.org/web/20160530160330/https://www.googl...