←back to thread

Why language models hallucinate

(openai.com)
277 points simianwords | 1 comments | 06 Sep 25 07:41 UTC | HN request time: 0s | source
Show context
thomasboyer ◴[06 Sep 25 13:50 UTC] No.45149259[source]▶
Great post. Teaching the models to doubt, to say "I don't know"/"I'm unsure"/"I'm sure" is a nice way to make them much better.
replies(2): >>45149845 #>>45150304 #
meshugaas ◴[06 Sep 25 15:46 UTC] No.45150304[source]▶
Look at their stats though. If they did this, more than half of responses would end up as “I don’t know.” Nobody would use something that did that.
replies(1): >>45150897 #
skybrian ◴[06 Sep 25 16:55 UTC] No.45150897[source]▶
It seems like it would train users to ask questions that it can actually answer. (They might also need some examples of what sort of questions to ask.)
replies(1): >>45151384 #
1. Jensson ◴[06 Sep 25 17:49 UTC] No.45151384[source]▶
Mostly it would train users to not use their service and go to a service where the model outputs results they can copy paste to complete their assignment.

So these companies cannot do this, they would hemorrhage too many users and companies cannot go against the profit incentives in practice.