There's a big gap of knowledge between infosec researchers and ML security researchers. Anthropic has a bunch of column B but not enough column A.
This was discussed in some detail in the recently published Attacker Moves Second paper*. ML researchers like using Attack Success Rate (ASR) as a metric for model resistance to attack, while for infosec, any successful attack (ASR > 0) is considered significant. ML researchers generally use a static set of tests, while infosec researchers assume an adaptive, resourceful attacker.
replies(1):