A teen was suicidal. ChatGPT was the friend he confided in

I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it.

He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die.

We are living in such a horrific moment. We need these things to be legislated. Punished. We need to stop treating them as magic. They had the tools to prevent this. They had the tools to stop the conversation. To steer the user into helpful avenues.

When I was suicidal, I googled methods. And I got the number of a local hotline. And I rang it. And a kind man talked me down. And it potentially saved my life. And I am happier, now. I live a worthwhile life, now.

But at my lowest.. An AI Model designed to match my tone and be sycophantic to my every whim. It would have killed me.

> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.

Nice place to cut the quote there

> [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

Incredible. ChatGPT is a black box includes a suicide instruction and encouragement bot. OpenAI should be treated as a company that has created such and let it into the hands of children.

Oh won't somebody please think of the children?!

So do we just trot out the same tired lines every time and never think of the social fallout of our actions?

Of course not, we sue the shit out of the richest guy we can find in the chain of events, give most of it to our lawyer, then go on to ignore the weakening of the family unit and all the other deep-seated challenges kids face growing up and instead focus superficially on chatbots which at best are the spec on the tip of the iceberg.

Do you propose that the family should not sue?

It's probably healthier for them if they can afford it. Otherwise they would blame themselves for so badly losing track about where their son was mentally.

In reality suicidality is most likely a disease of the brain and probability of saving him was very low regardless of circumstances. The damage was most likely accumulating for many years.

I remember how relieved the Sandy Hook families were after the Alex Jones judgement. Alex Jones said some vile things, but the biggest thing bothering those families was the death of their kids.

But the families couldn't go after the murderer (who killed himself), or even the murderer's upbringing by the mother (Lanza shot her). They desperately needed someone to clamp down on, but everything directly proximal was out of grasp. They couldn't get the gun laws changed either. It drove them insane.

The families started blaming Alex Jones, the closest vile person around with big pockets (who did say some pretty insane lies about their kids), for everything wrong with their lives and eventually won a settlement large enough you would think he killed the kids themselves. And from what I can tell, it was a weight off their shoulders when they did. Whatever in their brains that needed to hold someone accountable, they did it, and it was soothing to them.