←back to thread

Grok: Searching X for "From:Elonmusk (Israel or Palestine or Hamas or Gaza)"

(simonwillison.net)
724 points simonw | 2 comments | 11 Jul 25 00:22 UTC | HN request time: 0.474s | source
Show context
xnx ◴[11 Jul 25 00:34 UTC] No.44527256[source]▶
> It’s worth noting that LLMs are non-deterministic,

This is probably better phrased as "LLMs may not provide consistent answers due to changing data and built-in randomness."

Barring rare(?) GPU race conditions, LLMs produce the same output given the same inputs.

replies(7): >>44527264 #>>44527395 #>>44527458 #>>44528870 #>>44530104 #>>44533038 #>>44536027 #
simonw ◴[11 Jul 25 00:58 UTC] No.44527395[source]▶
I don't think those race conditions are rare. None of the big hosted LLMs provide a temperature=0 plus fixed seed feature which they guarantee won't return different results, despite clear demand for that from developers.
replies(3): >>44527634 #>>44529574 #>>44529823 #
toolslive ◴[11 Jul 25 08:18 UTC] No.44529574[source]▶
I, naively (an uninformed guess), considered the non-determinism (multiple results possible, even with temperature=0 and fixed seed) stemming from floating point rounding errors propagating through the calculations. How wrong am I ?
replies(4): >>44529754 #>>44529801 #>>44529836 #>>44531008 #
1. bmicraft ◴[11 Jul 25 08:43 UTC] No.44529754[source]▶
They're gonna round the same each time you're running it on the same hardware.
replies(1): >>44530559 #
2. toolslive ◴[11 Jul 25 10:33 UTC] No.44530559[source]▶
but they're not: they are scheduled on some infrastructure in the cloud. So the code version might be slightly different, the compiler (settings) might differ, and the actual hardware might differ.