←back to thread

724 points simonw | 3 comments | | HN request time: 0s | source
Show context
xnx ◴[] No.44527256[source]
> It’s worth noting that LLMs are non-deterministic,

This is probably better phrased as "LLMs may not provide consistent answers due to changing data and built-in randomness."

Barring rare(?) GPU race conditions, LLMs produce the same output given the same inputs.

replies(7): >>44527264 #>>44527395 #>>44527458 #>>44528870 #>>44530104 #>>44533038 #>>44536027 #
msgodel ◴[] No.44527264[source]
I run my local LLMs with a seed of one. If I re-run my "ai" command (which starts a conversation with its parameters as a prompt) I get exactly the same output every single time.
replies(2): >>44527284 #>>44527453 #
xnx ◴[] No.44527284[source]
Yes. This is what I was trying to say. Saying "It’s worth noting that LLMs are non-deterministic" is wrong and should be changed in the blog post.
replies(3): >>44527462 #>>44528765 #>>44529031 #
TheDong ◴[] No.44528765[source]
> Saying "It’s worth noting that LLMs are non-deterministic" is wrong and should be changed in the blog post.

Every person in this thread understood that Simon meant "Grok, ChatGPT, and other common LLM interfaces run with a temperature>0 by default, and thus non-deterministically produce different outputs for the same query".

Sure, he wrote a shorter version of that, and because of that y'all can split hairs on the details ("yes it's correct for how most people interact with LLMs and for grok, but _technically_ it's not correct").

The point of English blog posts is not to be a long wall of logical prepositions, it's to convey ideas and information. The current wording seems fine to me.

The point of what he was saying was to caution readers "you might not get this if you try to repro it", and that is 100% correct.

replies(2): >>44529058 #>>44530499 #
1. root_axis ◴[] No.44529058[source]
Still, the statement that LLMs are non-deterministic is incorrect and could mislead some people who simply aren't familiar with how they work.

Better phrasing would be something like "It's worth noting that LLM products are typically operated in a manner that produces non-deterministic output for the user"

replies(2): >>44529211 #>>44529618 #
2. Veen ◴[] No.44529211[source]
Simon would be less engaging if he caveated every generalisation in that way. It’s one of the main reasons academic writing is often tedious to read.
3. antonvs ◴[] No.44529618[source]
> It's worth noting that LLM products are typically operated in a manner that produces non-deterministic output for the user

Or you could abbreviate this by saying “LLMs are non-deterministic.” Yes, it requires some shared context with the audience to interpret correctly, but so does every text.