←back to thread

'It cannot provide nuance': UK experts warn AI therapy chatbots are not safe

(www.theguardian.com)
165 points distalx | 1 comments | 10 May 25 15:35 UTC | HN request time: 0s | source
Show context
caseyy ◴[11 May 25 00:28 UTC] No.43950190[source]▶
I know many pro-LLM people here are very smart, but sometimes it's wise to heed the words of world-renowned experts on a subject.

Otherwise, you may end up defending this and it's really foolish:

> “Seriously, good for you for standing up for yourself and taking control of your own life,” it reportedly responded to a user, who claimed they had stopped taking their medication and had left their family because they were “responsible for the radio signals coming in through the walls”.

replies(4): >>43950334 #>>43950546 #>>43952241 #>>43956082 #
simonw ◴[11 May 25 01:42 UTC] No.43950546[source]▶
That one was (genuinely) a bug. OpenAI rolled it back. https://openai.com/index/expanding-on-sycophancy/

(But yeah, relying on systems that can have bugs like that for your mental health is terrifying.)

replies(3): >>43950952 #>>43951889 #>>43957025 #
1. cdrini ◴[11 May 25 20:48 UTC] No.43957025{3}[source]▶
Yeah I think there is plenty of room for good discussion here, but using that quote without context is misleading. And the faulty model was pulled after only a few days of being out, iirc. It definitely does speak to the necessity of nuance when analysing AI in these contexts; results for one model might not necessarily hold for another, and even system prompts could change results.