←back to thread

Grok: Searching X for "From:Elonmusk (Israel or Palestine or Hamas or Gaza)"

(simonwillison.net)
724 points simonw | 1 comments | 11 Jul 25 00:22 UTC | HN request time: 1.317s | source
Show context
xnx ◴[11 Jul 25 00:34 UTC] No.44527256[source]▶
> It’s worth noting that LLMs are non-deterministic,

This is probably better phrased as "LLMs may not provide consistent answers due to changing data and built-in randomness."

Barring rare(?) GPU race conditions, LLMs produce the same output given the same inputs.

replies(7): >>44527264 #>>44527395 #>>44527458 #>>44528870 #>>44530104 #>>44533038 #>>44536027 #
simonw ◴[11 Jul 25 00:58 UTC] No.44527395[source]▶
I don't think those race conditions are rare. None of the big hosted LLMs provide a temperature=0 plus fixed seed feature which they guarantee won't return different results, despite clear demand for that from developers.
replies(3): >>44527634 #>>44529574 #>>44529823 #
1. diggan ◴[11 Jul 25 08:53 UTC] No.44529823[source]▶
> despite clear demand for that from developers

Theorizing about why that is: Could it be possible they can't do deterministic inference and batching at the same time, so the reason we see them avoiding that is because that'd require them to stop batching which would shoot up costs?