←back to thread

The "confident idiot" problem: Why AI needs hard rules, not vibe checks

(steerlabs.substack.com)
323 points steerlabs | 3 comments | 04 Dec 25 20:48 UTC | HN request time: 0.382s | source
Show context
jqpabc123 ◴[04 Dec 25 21:38 UTC] No.46153440[source]▶
We are trying to fix probability with more probability. That is a losing game.

Thanks for pointing out the elephant in the room with LLMs.

The basic design is non-deterministic. Trying to extract "facts" or "truth" or "accuracy" is an exercise in futility.

replies(17): >>46155764 #>>46191721 #>>46191867 #>>46191871 #>>46191893 #>>46191910 #>>46191973 #>>46191987 #>>46192152 #>>46192471 #>>46192526 #>>46192557 #>>46192939 #>>46193456 #>>46194206 #>>46194503 #>>46194518 #
1. hbs18 ◴[08 Dec 25 14:59 UTC] No.46192939[source]▶
> The basic design is non-deterministic

Is it? I thought an LLM was deterministic provided you run the exact same query on exact same hardware at a temperature of 0.

replies(2): >>46193272 #>>46194041 #
2. chmod775 ◴[08 Dec 25 15:24 UTC] No.46193272[source]▶
Not quite then as well, since a lot is typically executed in parallel and the implementation details of most number representations make them sensitive to the order of operations.

Given how much number crunching is at the heart of LLMs, these small differences add up.

3. biophysboy ◴[08 Dec 25 16:13 UTC] No.46194041[source]▶
My understanding is that it selects from a probability distribution. Raising the temperature merely flattens that distribution, Boltzmann factor style