←back to thread

165 points distalx | 3 comments | | HN request time: 0.512s | source
Show context
caseyy ◴[] No.43950190[source]
I know many pro-LLM people here are very smart, but sometimes it's wise to heed the words of world-renowned experts on a subject.

Otherwise, you may end up defending this and it's really foolish:

> “Seriously, good for you for standing up for yourself and taking control of your own life,” it reportedly responded to a user, who claimed they had stopped taking their medication and had left their family because they were “responsible for the radio signals coming in through the walls”.

replies(4): >>43950334 #>>43950546 #>>43952241 #>>43956082 #
simonw ◴[] No.43950546[source]
That one was (genuinely) a bug. OpenAI rolled it back. https://openai.com/index/expanding-on-sycophancy/

(But yeah, relying on systems that can have bugs like that for your mental health is terrifying.)

replies(3): >>43950952 #>>43951889 #>>43957025 #
1. vrighter ◴[] No.43951889[source]
you cannot really roll back a bug in a black box system you don't understand
replies(2): >>43952220 #>>43957048 #
2. clncy ◴[] No.43952220[source]
Exactly. More like changing the state of the system to reduce the observed behaviour while introducing other (unknown) behaviours
3. cdrini ◴[] No.43957048[source]
I don't think that's quite true; even a system you don't understand has observable behaviour. And you can roll back to a certain state that doesn't exhibit the undesirable observable behaviour. If anything, most things in life operate this way.