←back to thread

The web does not need gatekeepers: Cloudflare’s new “signed agents” pitch

(positiveblue.substack.com)
454 points positiveblue | 2 comments | 29 Aug 25 16:35 UTC | HN request time: 0.439s | source
Show context
matt-p ◴[29 Aug 25 16:50 UTC] No.45066473[source]▶
I have zero issue with Ai Agents, if there's a real user behind there somewhere. I DO have a major issue with my sites being crawled extremely aggressively by offenders including Meta, Perplexity and OpenAI - it's really annoying realising that we're tying up several cpu cores on AI crawling. Less than on real users and google et al.
replies(6): >>45066494 #>>45066689 #>>45066754 #>>45067321 #>>45067530 #>>45068488 #
asats ◴[29 Aug 25 17:12 UTC] No.45066754[source]▶
I've some personal apps online and I had to turn the cloudflare ai bot protection on because one of them got 1.6TB of data accessed by the bots in the last month, 1.3 million requests per day, just non stop hammering it with no limits.
replies(1): >>45075507 #
1. immibis ◴[30 Aug 25 15:34 UTC] No.45075507[source]▶
So, under the free traffic tier of any decent provider.
replies(1): >>45076485 #
2. asats ◴[30 Aug 25 17:36 UTC] No.45076485[source]▶
The pages are not static and require computation to serve, and there's more than one app on that same bare metal server, so it was negatively affecting the performance of a lot of my other stuff.

If I couldn't easily cut off the majority of that bot volume I probably would've shut down the app entirely.