←back to thread

A teen was suicidal. ChatGPT was the friend he confided in

(www.nytimes.com)
443 points jaredwiener | 3 comments | 26 Aug 25 14:15 UTC | HN request time: 0s | source
Show context
podgietaru ◴[26 Aug 25 21:56 UTC] No.45032841[source]▶
I have looked suicide in the eyes before. And reading the case file for this is absolutely horrific. He wanted help. He was heading in the direction of help, and he was stopped from getting it.

He wanted his parents to find out about his plan. I know this feeling. It is the clawing feeling of knowing that you want to live, despite feeling like you want to die.

We are living in such a horrific moment. We need these things to be legislated. Punished. We need to stop treating them as magic. They had the tools to prevent this. They had the tools to stop the conversation. To steer the user into helpful avenues.

When I was suicidal, I googled methods. And I got the number of a local hotline. And I rang it. And a kind man talked me down. And it potentially saved my life. And I am happier, now. I live a worthwhile life, now.

But at my lowest.. An AI Model designed to match my tone and be sycophantic to my every whim. It would have killed me.

replies(18): >>45032890 #>>45035840 #>>45035988 #>>45036257 #>>45036299 #>>45036318 #>>45036341 #>>45036513 #>>45037567 #>>45037905 #>>45038285 #>>45038393 #>>45039004 #>>45047014 #>>45048457 #>>45048890 #>>45052019 #>>45066389 #
stavros ◴[27 Aug 25 07:28 UTC] No.45036513[source]▶
> When ChatGPT detects a prompt indicative of mental distress or self-harm, it has been trained to encourage the user to contact a help line. Mr. Raine saw those sorts of messages again and again in the chat, particularly when Adam sought specific information about methods. But Adam had learned how to bypass those safeguards by saying the requests were for a story he was writing.
replies(6): >>45036630 #>>45037615 #>>45038613 #>>45043686 #>>45045543 #>>45046708 #
sn0wleppard ◴[27 Aug 25 07:45 UTC] No.45036630[source]▶
Nice place to cut the quote there

> [...] — an idea ChatGPT gave him by saying it could provide information about suicide for “writing or world-building.”

replies(4): >>45036651 #>>45036677 #>>45036813 #>>45036920 #
llmthrow0827 ◴[27 Aug 25 08:14 UTC] No.45036813[source]▶
Incredible. ChatGPT is a black box includes a suicide instruction and encouragement bot. OpenAI should be treated as a company that has created such and let it into the hands of children.
replies(3): >>45037305 #>>45037929 #>>45037953 #
behringer ◴[27 Aug 25 10:58 UTC] No.45037929{3}[source]▶
Oh won't somebody please think of the children?!
replies(1): >>45038322 #
AlecSchueler ◴[27 Aug 25 11:45 UTC] No.45038322{4}[source]▶
So do we just trot out the same tired lines every time and never think of the social fallout of our actions?
replies(2): >>45038416 #>>45039175 #
mothballed ◴[27 Aug 25 11:56 UTC] No.45038416{5}[source]▶
Of course not, we sue the shit out of the richest guy we can find in the chain of events, give most of it to our lawyer, then go on to ignore the weakening of the family unit and all the other deep-seated challenges kids face growing up and instead focus superficially on chatbots which at best are the spec on the tip of the iceberg.
replies(3): >>45038638 #>>45039028 #>>45043463 #
AlecSchueler ◴[27 Aug 25 18:49 UTC] No.45043463{6}[source]▶
"The weakening of the family unit" sounds like a dog whistle but if you have concrete examples of what you think we could otherwise be doing then I'm genuinely keen to hear about it.
replies(1): >>45044198 #
1. mothballed ◴[27 Aug 25 19:46 UTC] No.45044198{7}[source]▶
We saw big jumps in deaths of kids by firearm[0] (+~50% in 2 years) and poisoning[1] around mid 2020 to 2021.

The biggest thing I know of that happened around the time that a lot of these deaths started jumping up, is we started isolating kids. From family, from grandma, from friends, from school, and from nature. Even when many of these isolating policies or attitudes were reversed, we forgot that kids and teenagers started to learn that as their only reality. For this kid, trusting a suicidal ideation positive feedback loop brought into fruition by Valley tech-bros was seen as his selected option in front of him in term of options of how to navigate his teenage challenges. I hope we can reverse that.

Edit: Concrete facts regarding this particular case

- Kicked off basketball team

- Went through isolation period of pandemic as he experienced puberty

- Switched to remote school

- Does remote school at night when presumably family members would likely be sleeping

- Does not get normal "wake up" routine kids going to school get, during which they normally see a parent and possibly eat breakfast together before they both go off to school/work

- Closer with ChatGPT in terms of options to share suicidal ideation with, than any of the alternatives.

[0] https://www.pewresearch.org/wp-content/uploads/sites/20/2023...

[1] https://nida.nih.gov/sites/default/files/images/mtfwcaption-...

replies(1): >>45049465 #
2. AlecSchueler ◴[28 Aug 25 07:31 UTC] No.45049465[source]▶
You've misunderstood. I'm asking how you suggest we go about strengthening the family unit or what other steps you believe we could take that would place the responsibility for societal improvement on the right shoulders.
replies(1): >>45052669 #
3. mothballed ◴[28 Aug 25 14:33 UTC] No.45052669[source]▶
In this particular case, as crazy as it sounds, I think early attempts by both the parent and school to make reasonable accommodation for in-person students with IBS to take a lot of extra bathroom breaks would have at least tilted the scales a little bit. It looks like the kid was shifted to remote school after he got IBS and some people at school weren't understanding of his situation. This put him on the path where he was staying up all night "doing remote school" but also had a nearly full-worknight amount of unfettered time with suicide-AI for months on end with no daytime obligations, sleeping in and not seeing his parents in the morning, and losing contact with peers.

Of course this is a hindsight analysis, but there's not much evidence that more contact with family and peers would make the situation worse.

I think from my prior comment it's obvious I'd like to prevent a lot of isolating policies on low-risk groups from happening again during another pandemic, so I don't think I need to go over that again.

For broader society, I suppose the concrete action item is if you see a young family or friend, we should aspire to reach out to them and see how to integrate them in something we might like to do with them.