←back to thread

Web Bot Auth

(developers.cloudflare.com)
81 points ananddtyagi | 1 comments | | HN request time: 0.214s | source
Show context
bobbiechen ◴[] No.45056701[source]
I disagree with the other top-level comments at the moment: I believe Web Bot Auth is a useful and non-centralized emerging standard for self-identifying bots and agents.

This press release today is a better statement of _why_ this feature exists (as opposed to the submission link, which is nuts-and-bolts of implementing): https://blog.cloudflare.com/signed-agents/

Web Bot Auth is a way for bots to self-identify cryptographically. Unlike the user agent header (which is trivially spoofed) or known IPs (painful to manage), Web Bot Auth uses HTTP Message Signatures using the bot's key, which should be published at some well-known location.

This is a good thing! We want bots to be able to self-identify in a way that can't be impersonated. This gives website operators the power to allow or deny well-behaved bots with precision. It doesn't change anything about bots who try to hide their identity, who are not going to self-identify anyways.

It's worth reading the proposal on the details: https://datatracker.ietf.org/doc/html/draft-meunier-web-bot-... . Nothing about this is limited to Cloudflare.

I'm also working on support for Web Bot Auth for our Agent Identification project at Stytch https://www.isagent.dev . Well-behaved bots benefit from this self-identification because it enables a better Agent Experience: https://stytch.com/blog/introducing-is-agent/

replies(6): >>45056742 #>>45057086 #>>45060053 #>>45061067 #>>45062265 #>>45062681 #
1. sneak ◴[] No.45060053[source]
The problem with this is that key generation is free, so being a well-behaved unknown bot is the same as being an unidentified bot, which means that you go in the block/captcha/throttle bucket.

It is only useful for whitelisting bots, not for banning bad ones, as bad ones can rotate keys.

Whitelisting clients by identity is the death of the open web, and means that nobody will ever be able to compete with capital on even footing.