Anthropic’s paper smells like bullshit

There is absolutely no way a technical person would mix those up

Right! It's well known that technical people never make mistakes.

I think the expectation is more that serious people have their work checked over by other serious people to catch the obvious mistakes.

But what about an ML person roped into writing an AI assisted blogpost about security

Every time you have your work "checked over by other serious people", it eliminates 90% of the mistakes. So you have it checked over twice so that 99% of mistakes have been eliminated, and so on. But it never gets to 0% mistakes. That's my experience anyway.

Serious people like to look at things through a magnifying glass. Which makes them miss a lot.

I've seen printed books checked by paid professionals that consisted a "replace all" populated without context. Creating a grammar error on every single page. Or ones where everyone just forgot to add page numbers. Or a large cook book where index and page numbers didn't mach, making it almost impossible to navigate.

I'm talking of pre-AI work, with publisher. Apparently it wasn't obvious for them.

Every time you have your work "checked over by other serious people", it only means it's been checked over by other people. You can't attach a metric to this process. Especially when it comes to security, adding more internal eyeballs doesn't mean you've expanded coverage.

One of the things I enjoy about Penn and Teller is that they explain in detail how their point of view differs from the audiences and how they intentionally use that difference in their shows. With that in mind you might picture your org as the audience, with one perspective diligently looking forwards.