Given that this is 'research' carried out (and seemingly published) by a company with a direct interest in selling you a product (or, rather, getting investors excited/panicked), can we trust it?
replies(6):
I think Anthropic genuinely cares about model welfare and wants to make sure they aren't spawning consciousness, torturing it, and then killing it.
For anyone having paid attention, it has been clear for the past two years that Dario Amodei is lobbying for strict regulation on LLMs to prevent new entrants on the market, and the core of its argument is that LLMs are fundamentally intelligent and dangerous.
So this kind of “research” isn't targeted towards their customers but towards the legislators.
- it's a threat for young graduates' jobs.
- it's a threat to the school system, undermining its ability to teach through exercises.
- it's a threat to the internet given how easily it can create tons of fake content.
- it's a threat to mental health of fragile people.
- it's a gigantic threat to a competitive economy if all the productivity gains are being grabbed by the AI editors through a monopolistic position.
The terminator threat is pure fantasy and it's just here to distract from the very real threats that are already doing harm today.