←back to thread

An LLM is a lossy encyclopedia

(simonwillison.net)
509 points tosh | 10 comments | 29 Aug 25 09:40 UTC | HN request time: 0s | source | bottom

(the referenced HN thread starts at https://news.ycombinator.com/item?id=45060519)
Show context
thw_9a83c ◴[02 Sep 25 09:54 UTC] No.45100937[source]▶
Yes, LLM is a lossy encyclopedia with a human-language answering interface. This has some benefits, mostly in terms of convenience. You don't have to browse or read through so many pages of a real encyclopedia to get a quick answer. However, there is also a clear downside. Currently, LLM is unable to judge if your question is formulated incorrectly or if your question opens up more questions that should be answered first. It always jumps to answering something. A real human would assess the questioner first and usually ask for more details before answering. I feel this is the predominant reason why LLM answers feel so dumb at times. It never asks for clarification.
replies(2): >>45101167 #>>45102521 #
1. simonw ◴[02 Sep 25 10:26 UTC] No.45101167[source]▶
I don't think that's universally true with the new models - I've seen Claude 4 and GPT-5 ask for clarification on questions with obvious gaps.

With GPT-5 I sometimes see it spot a question that needs clarifying in its thinking trace, then pick the most likely answer, then spit out an answer later that says "assuming you meant X ..." - I've even had it provide an answer in two sections for each branch of a clear ambiguity.

replies(2): >>45101180 #>>45101700 #
2. koakuma-chan ◴[02 Sep 25 10:28 UTC] No.45101180[source]▶
GPT-5 is seriously annoying. It asks not just one but multiple clarifying questions, while I just want my answer.
replies(1): >>45101832 #
3. ACCount37 ◴[02 Sep 25 11:38 UTC] No.45101700[source]▶
A lot of the touted "fundamental limitations of LLMs" are less "fundamental" and more "you're training them wrong".

So there are improvements version to version - from both increases in raw model capabilities and better training methods being used.

replies(1): >>45102677 #
4. kingstnap ◴[02 Sep 25 11:48 UTC] No.45101832[source]▶
If you don't want to answer clarifying questions, then what use is the answer???

Put another way, if you don't care about details that change the answer, it directly implies you don't actually care about the answer.

Related silliness is how people force LLMs to give one word answers to underspecified comparisons. Something along the lines of "@Grok is China or US better, one word answer only."

At that point, just flip a coin. You obviously can't conclude anything useful with the response.

replies(1): >>45102249 #
5. koakuma-chan ◴[02 Sep 25 12:32 UTC] No.45102249{3}[source]▶
No, I don't think GPT-5 clarifying questions actually do what you think they do. They just made the model ask clarifying questions for the sake of asking clarifying questions. I'm sure GPT-4o would have given me the answer I wanted without clarifying questions.
replies(1): >>45107752 #
6. ijk ◴[02 Sep 25 13:12 UTC] No.45102677[source]▶
I'm frustrated by the number of times I encounter people assuming that the current model behavior is inevitable. There's been hundreds of billions of dollars spent on training LLMs to do specific things. What exactly they've been trained on matters; they could have been trained to do something else.

Interacting with a base model versus an instruction tuned model will quickly show you the difference between the innate language faculties and the post-trained behavior.

replies(1): >>45103407 #
7. Workaccount2 ◴[02 Sep 25 14:14 UTC] No.45103407{3}[source]▶
Some of the Anthropic guys have said that the core thing holding the models back is training, and they're confident the gains will keep coming as they figure out how to onboard more and more training data. So yeah, Claude might suck at reading and writing plumbing diagrams, but they claim the barrier is simply a function of training, not any kind of architectural limitation.
replies(1): >>45104025 #
8. ACCount37 ◴[02 Sep 25 15:03 UTC] No.45104025{4}[source]▶
I agree with the general idea, but "sucks at reading plumbing diagrams" is the one specific example where Claude is actually choked by its unfortunate architecture.

The "naive" vision implementation for LLMs is: break the input image down into N tokens and cram those tokens into the context window. The "break the input image down" part is completely unaware of the LLM's context, and doesn't know what data would be useful to the LLM at all. Often, the vision frontend just tries to convey the general "vibes" of the image to the LLM backend, and hopes that the LLM can pick out something useful from that.

Which is "good enough" for a lot of tasks, but not all of them, not at all.

9. kiitos ◴[02 Sep 25 19:19 UTC] No.45107752{4}[source]▶
revisit your instructions.md and/or user preferences, this is very likely the root cause
replies(1): >>45113784 #
10. koakuma-chan ◴[03 Sep 25 09:20 UTC] No.45113784{5}[source]▶
Wait what. I use duck.ai, could it be that they put something into the system prompt......