←back to thread

The "confident idiot" problem: Why AI needs hard rules, not vibe checks

(steerlabs.substack.com)
323 points steerlabs | 5 comments | 04 Dec 25 20:48 UTC | HN request time: 0.314s | source
Show context
jqpabc123 ◴[04 Dec 25 21:38 UTC] No.46153440[source]▶
We are trying to fix probability with more probability. That is a losing game.

Thanks for pointing out the elephant in the room with LLMs.

The basic design is non-deterministic. Trying to extract "facts" or "truth" or "accuracy" is an exercise in futility.

replies(17): >>46155764 #>>46191721 #>>46191867 #>>46191871 #>>46191893 #>>46191910 #>>46191973 #>>46191987 #>>46192152 #>>46192471 #>>46192526 #>>46192557 #>>46192939 #>>46193456 #>>46194206 #>>46194503 #>>46194518 #
1. sweezyjeezy ◴[08 Dec 25 13:27 UTC] No.46191973[source]▶
You could make an LLM deterministic if you really wanted to without a big loss in performance (fix random seeds, make MoE batching deterministic). That would not fix hallucinations.

I don't think using deterministic / stochastic as a diagnostic is accurate here - I think that what we're really talking is about some sort of fundamental 'instability' of LLMs a la chaos theory.

replies(3): >>46192124 #>>46192177 #>>46199061 #
2. rs186 ◴[08 Dec 25 13:43 UTC] No.46192124[source]▶
We talk about "probability" here because the topic is hallucination, not getting different answers each time you ask the same question. Maybe you could make the output deterministic but does not help with the hallucination problem at all.
replies(1): >>46192353 #
3. ajuc ◴[08 Dec 25 13:48 UTC] No.46192177[source]▶
Yeah deterministic LLMs just hallucinate the same way every time.
4. sweezyjeezy ◴[08 Dec 25 14:07 UTC] No.46192353[source]▶
Exactly - 'non-deterministic' is not an accurate diagnosis of the issue.
5. encyclopedism ◴[08 Dec 25 23:20 UTC] No.46199061[source]▶
Hallucinations can never be fixed. LLM's 'hallucinate' because that is literally what they can ONLY do, provide some output given some input. The output is measured and judged by a human who then classifies it as 'correct' or 'incorrect'. In the later case it seems to be labelled as a 'hallucination' as if it did something wrong. It did nothing wrong and worked exactly as it was programmed to do.