A teen was suicidal. ChatGPT was the friend he confided in

It says a lot about HN that a story like this has so much resistance getting any real traction here.

This sucks but the only solution is to make companies censor the models, which is a solution we all hate, so there’s that.

Thank you, “we just have to accept that these systems will occasionally kill children” is a perfect example of the type of mindset I was criticizing.

Don’t cars and ropes and drills occasionally kill people too? Society seems to have accepted that fact long ago.

Somehow we expect the digital world to be devoid of risks.

Cryptography that only the good guys can crack is another example of this mindset.

Now I’m not saying ClosedAI look good on this, their safety layer clearly failed and the sycophantic BS did not help.

But I reckon this kind of failure more will always exist in LLMs. Society will have to learn this just like we learned cars are dangerous.