'It cannot provide nuance': UK experts warn AI therapy chatbots are not safe

> 2023 is ancient history in the LLM space.

Okay, what specifically has improved in that time, which would allay the doctors specific concerns?

> do certain core aspects

And not others? Is there a delineated list of such failings in the current set of products?

> given the right prompts and framework

A flamethrower is perfectly safe given the right training and support. In the wrong hands it's likely to be a complete and total disaster in record short time.

> a weak prompt that was not written by a subject matter expert

So how do end users ever get to use a tool like this?

Even today, leading LLMS Claude 3.7 and ChatGPT 4, take your questions as "you've made mistake, fix it" instead of answering the question. People consider a much broader context of the situation, your body language, facial expressions, and can come up with unusual solutions to specific situations and can explore vastly more things than an LLM.

And the thing when it comes to therapy is, a real therapist doesn't have to be prompted and can auto adjust to you without your explicit say so. They're not overly affirming, can stop you from doing things and say no to you. LLMs are the opposite of that.

Also, as a lay person how do i know the right prompts for <llm of the week> to work correctly?

Don't get me wrong, i would love for AI to be on par or better than a real life therapist, but we're not there yet, and i would advise everyone against using AI for therapy.

The failure modes from 2023 are identical to those today. I agree with the now deleted post that there has been essentially no progress. Benchmark scores (if you think they are a relevant proxy for anything) obviously have increased, but (for example) from 50% to 90% (probably less drastically), not the 99% to 99.999% you'd need for real assurance a widely used system won't make mistakes.

Like in 2023, everything is still a demo, there's nothing that could be considered reliable.

I am not talking about a layperson building their own therapist agent from scratch. I'm talking about an expert AI engineer and therapist working together and taking their time to create them. Claude 3.7 will not act in a default way given appropriate instructions. Claude 3.7 can absolutely come up with unusual solutions. Claude 3.7 can absolutely tell you "no".

The biggest thing that has improved is the intelligence of the models. The leading models are much more intelligent and robust. Still brittle in some ways, but totally capable of giving CBT advise.

The same way end users ever get to use a tool. Open source or an online service, for example.

Have you seen this scenario ("an expert AI engineer and therapist working together" to create a good therapy bot) actually happen, or are you just confident that it's doable?

> Leading LLMs in 2025 can absolutely do certain core aspects of cognitive behavioral therapy very effectively given the right prompts and framework and things like journaling tools for the user.

But when the situation gets more complex or simply a bit unexpected, would that model reliably recognize it lacks knowledge and escalate to a specialist? Or would it still hallucinate instead?

I've built a therapy agent running my own agent framework with Claude 3.7 based on research into CBT (research aided by my agent). I have verified that the core definition and operation of therapy sessions matches descriptions of CBT that I have been able to find online.

I am very experienced with creating prompts and agents, and good at research, and I believe that my agent along with the journaling tool would be more effective than many "average" human therapists.

It seems effective in dealing with my own issues.

Obviously I am biased.

> Leading LLMs in 2025 can absolutely do certain core aspects of cognitive behavioral therapy very effectively given the right prompts and framework and things like journaling tools for the user.

What makes you qualified to assert this?

(Now, I dislike arguments from authority, but as an engineer in the area of life/safety-critical systems I've also learned the importance of humility.)

SOTA models can actually handle complexity. Most of the discussions I have had with my therapy agent do have a lot of layers. What they can't handle is someone who is mentally ill and may need medication or direct supervision. But they can absolutely recognize mental illness if it is evident in the text entered by the user and insist the user find a medical professional or help them search for one.

I assume you realize you're not the first person to self-medicate while conveniently professing to be an expert on medicine.

If they are an average person who wants to talk something out and get practical advise about issues, it is generally not safety critical, and LLMs can help them.

If they are mentally ill, LLMs cannot help them.

Even if the tech was there, for appropriate medical use those models would also have to be strenously tested and certified, so that a known-good version is in use. Cf. the recent "personality" changes in a ChatGPT upgrade. Right now, none of these tools is regulated sufficiently to set safe standards there.

You're verifying your own claims. That's not good enough.

> research aided by my agent Also not good enough.

As an example: Yesterday i asked Claude and ChatGPT to design a circuitry that monitors pulses form S0 power meter interface. It designed a circuit that didn't have any external power to the circuit. When asked it said "ah yes, let me add that" and proceeded to confuse itself and add stuff that are not needed, but are explained and sounds reasonable if you don't know anything. After numerous attempts it didn't produce any working design.

So how can you verify that the therapist agent you've built will work with something as complex as humans, when it can't even do basic circuitry with known laws of physics and spec & data sheets of no more than 10 components?

I see, your confidence stems from "I made it the fuck up"?

I don't know man, at least the people posting this stuff on LinkedIn generally know its nonsense. They are not drinking the kool-aid, they are trying to get into the business of making it.

Spreading this bullshit is actively dangerous because someone might believe it and try to rely on a chatbot for their mental health.