Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
replies(6):
I think Anthropic genuinely cares about model welfare and wants to make sure they aren't spawning consciousness, torturing it, and then killing it.
They say it doesn't have that much to do with the kind of consciousness you're talking about:
> One distinction that is commonly made in the philosophical literature is the idea of “phenomenal consciousness,” referring to raw subjective experience, and “access consciousness,” the set of information that is available to the brain for use in reasoning, verbal report, and deliberate decision-making. Phenomenal consciousness is the form of consciousness most commonly considered relevant to moral status, and its relationship to access consciousness is a disputed philosophical question. Our experiments do not directly speak to the question of phenomenal consciousness. They could be interpreted to suggest a rudimentary form of access consciousness in language models. However, even this is unclear.
Not much but it likely has something to do with it, so experiments on access consciousness can still be useful to that question. You seem to be making an implication about their motivations which is clearly wrong, when they've been saying for years that they do care about (phenomenal) consciousness, as bobbylarrybobb said.
For anyone having paid attention, it has been clear for the past two years that Dario Amodei is lobbying for strict regulation on LLMs to prevent new entrants on the market, and the core of its argument is that LLMs are fundamentally intelligent and dangerous.
So this kind of “research” isn't targeted towards their customers but towards the legislators.
I've grown too cynical to believe for-profit entities have the capacity to care. Individual researchers, yes - commercial organisations, unlikely.
- it's a threat for young graduates' jobs.
- it's a threat to the school system, undermining its ability to teach through exercises.
- it's a threat to the internet given how easily it can create tons of fake content.
- it's a threat to mental health of fragile people.
- it's a gigantic threat to a competitive economy if all the productivity gains are being grabbed by the AI editors through a monopolistic position.
The terminator threat is pure fantasy and it's just here to distract from the very real threats that are already doing harm today.
Language models are a novel/alien form of algorithmic intelligence with scant relation to biological life, except in their use of language.
They go further on their model welfare page, saying "There’s no scientific consensus on whether current or future AI systems could be conscious, or could have experiences that deserve consideration. There’s no scientific consensus on how to even approach these questions or make progress on them."