'It cannot provide nuance': UK experts warn AI therapy chatbots are not safe

I know many pro-LLM people here are very smart, but sometimes it's wise to heed the words of world-renowned experts on a subject.

Otherwise, you may end up defending this and it's really foolish:

> “Seriously, good for you for standing up for yourself and taking control of your own life,” it reportedly responded to a user, who claimed they had stopped taking their medication and had left their family because they were “responsible for the radio signals coming in through the walls”.

As much as I tend to defer to experts, you must also be weary of experts whose very livelihoods are at risk. They may not have your interests at heart.

Hell yeah, rail against those profiteering…therapists.

Man I hate this modern shift of “actually anyone who is an expert is also trying to deceive me”. Extremely healthy shit for a civilization.

Is there something about therapists that makes them inherently noble and not prone to the same incentives as everyone else?

The implication was that the low salary selects for people who value helping people more than they value money.

Experts have a direct and obvious inventive to justify their existence. Radio experts warned us about TV. TV experts warned us about the Internet. If you live long enough you see it over and over again

Indeed says that therapists in Toronto, Canada make about CAD $55/hour. That's not FAANG level nor what you'd expect for an MD, but it's not what I'd call low, either.

That said, I certainly don't see therapists as profiteering, in the sense of trying to convince people to pay for therapy they don't need. They might plausibly feel threatened by AI, but they'd absolutely be justified in calling out examples like those in TFA.

The existence of a similar contradictory example does not disprove the original point. It's okay to be suspicious and cynical, but nuance is still important.

Assuming that anyone who has anything to gain by you believing them is out to get you is rash and leads you only to those who are more willing to lie about their motivations. Yes, doctors and Big Pharma (tm) are financially motivated to sell you cures, but the guy selling you a juice cleanse ""at cost"" for your cancer is still not trustworthy.

That one was (genuinely) a bug. OpenAI rolled it back. https://openai.com/index/expanding-on-sycophancy/

(But yeah, relying on systems that can have bugs like that for your mental health is terrifying.)

It's terrifying that this applies to doctors, teachers, firefighters, and entrepreneurs.

We were not talking about engineers so thanks for your strawman

> Yes, doctors and Big Pharma (tm) are financially motivated to sell you cures, but the guy selling you a juice cleanse ""at cost"" for your cancer is still not trustworthy.

Two things can be true at the same time. Don't trust anyone because nobody is transparent about their incentives. Your doctor does not disclose to you that they were at a congress in Hawaii for company X when he prescribes you to use company X drug for your ailment.

Maybe we live in different worlds. I see all of those professions justifying their existence. That's exactly why we should distrust them. They have an incentive to do so.

> TV experts warned us about the Internet

If they warned that it could become a distillation of the worst aspects of television... maybe they weren't wrong.

The implication is that it's insane to simply apply a "everyone wants to grift from you" angle to anyone who as an expert without any evidence or analysis.

Brother you just described every worker.

you cannot really roll back a bug in a black box system you don't understand

I didn't say "grift". Of course therapists are going to warn you against technology that replaces therapists for a fraction of the cost, regardless of how effective it is. That's just human nature. There's nothing wrong with self-preservation, we just need to be on the lookout for it.

This reads like an attempt to restate the buggy whip idiom. It doesn't work well even though your point has some merit.

Exactly. More like changing the state of the system to reduce the observed behaviour while introducing other (unknown) behaviours

The very fact that the "world class experts" are warning people not to use it means they have already been replaced in most fields that matter.

They didn't feel threatened by systems like cleverbot or GPT-3.5

And the tech bros pushing magic chatbots that they nor anyone else has any insight into but from which the same tech bros derive an even higher salary and more opulent livelihood and collect additional rent from certainly do have your interests at heart?

Fuck me. Maybe that guy on the street corner selling salvation or “cuckane” really was dealing in the real thing, too, eh?

$55/hr in Toronto… not to mention the overhead to pay for the building and support staff, the fact that you can’t bill for 40 hours per week, and the mentally demanding nature of the job listening to people’s mental shit all day.

Hot take: Therapists should earn more than most software devs.

Given the incredible gap between demand and supply in many places, I think many therapists would welcome a stopgap solution for people on waiting lists or struggling with costs. And they would not feel their livelihoods threatened one bit. That is, if they trusted that stopgap to at worst do no harm.

Congrats, you have trapped yourself in an ideological bubble where nobody can ever tell you that AI is a bad fit for a given application.

Try this on for size: I am not a therapist, but I will happily tell you that a statistical word generating LLM is a truly atrocious substitute for the hard work of a creative, empathetic and caring human being.

There's an inherent limit in how many people they can be treating - even group therapy sessions will be limited by number. As such, there's not many "exploits" that they can use to gain more and more power/money. Also, the job is far more likely to attract people that are interested in helping rather than exploiting people. People that want to exploit others are going to want to expand their audience.

Yeah I think there is plenty of room for good discussion here, but using that quote without context is misleading. And the faulty model was pulled after only a few days of being out, iirc. It definitely does speak to the necessity of nuance when analysing AI in these contexts; results for one model might not necessarily hold for another, and even system prompts could change results.

I don't think that's quite true; even a system you don't understand has observable behaviour. And you can roll back to a certain state that doesn't exhibit the undesirable observable behaviour. If anything, most things in life operate this way.

Therapists are definitely the reason for many software devs not burning out. Or burning out slowly enough to earn a retirement.

Hmm, how is that calculated? Therapist I'm currently seeing is $150/hour, on the outskirts of Toronto (cheaper cost of living / office space), and is empathically not on the high end of training (i.e. no PhD etc). I've never ever seen one below triple digits, so the notion of average, in Toronto, being $55 doesn't seem right.

Ahh, here we go; Indeed.com includes "Stretch Therapist", Occupational Therapist, Respiratory Therapist, Weight Gain Therapist, Massage Therapist, Care Co-ordinator, and everything else that includes the term "Therapist" in it.