←back to thread

Claude for Chrome

(www.anthropic.com)
795 points davidbarker | 1 comments | | HN request time: 0.204s | source
Show context
stusmall ◴[] No.45033056[source]
It's wild to see an AI company put out a press release that is basically "hey, you kids wanna see a loaded gun?" Normally all their public coms are so full of optimism and salesmanship around the potential. They are fully aware of how dangerous this is.
replies(8): >>45033105 #>>45033148 #>>45033197 #>>45033279 #>>45033315 #>>45033347 #>>45033852 #>>45037231 #
asdff ◴[] No.45033279[source]
> "We conducted extensive adversarial prompt injection testing, evaluating 123 test cases representing 29 different attack scenarios. "

Doesn't this seem like a remarkably small set of tests? And the fact that it took this testing to realize that prompt injection and giving the reigns to the AI agent is dangerous strikes me as strange that this wasn't anticipated while building the tool in the first place, before it even went to their red team.

Move fast and break things I guess. Only it is the worlds largest browser and the risk of breaking things means financial ruin and/or the end of the internet as we know it as a human to human communication tool.

replies(2): >>45033455 #>>45041764 #
1. fwip ◴[] No.45041764[source]
And even after their mitigations on known attacks, the attacks were still successful 11% of the time!

To misquote the IRA - "[Scammers] only need to be lucky once, you need to be lucky every time." Even a 1% chance of getting pwned every time you get sent a malicious email is way too high. Plus the scammers aren't gonna rest on their laurels - they'll be iterating too.