←back to thread

An LLM is a lossy encyclopedia

(simonwillison.net)
509 points tosh | 3 comments | 29 Aug 25 09:40 UTC | HN request time: 0s | source

(the referenced HN thread starts at https://news.ycombinator.com/item?id=45060519)
Show context
latexr ◴[02 Sep 25 10:27 UTC] No.45101170[source]▶
A lossy encyclopaedia should be missing information and be obvious about it, not making it up without your knowledge and changing the answer every time.

When you have a lossy piece of media, such as a compressed sound or image file, you can always see the resemblance to the original and note the degradation as it happens. You never have a clear JPEG of a lamp, compress it, and get a clear image of the Milky Way, then reopen the image and get a clear image of a pile of dirt.

Furthermore, an encyclopaedia is something you can reference and learn from without a goal, it allows you to peruse information you have no concept of. Not so with LLMs, which you have to query to get an answer.

replies(10): >>45101190 #>>45101267 #>>45101510 #>>45101793 #>>45101924 #>>45102219 #>>45102694 #>>45104357 #>>45108609 #>>45112011 #
Lerc ◴[02 Sep 25 13:14 UTC] No.45102694[source]▶
The argument is that a banana is a squishy hammer.

You're saying hammers shouldn't be squishy.

Simon is saying don't use a banana as a hammer.

replies(1): >>45103574 #
latexr ◴[02 Sep 25 14:27 UTC] No.45103574[source]▶
> You're saying hammers shouldn't be squishy.

No, that is not what I’m saying. My point is closer to “the words chosen to describe the made up concept do not translate to the idea being conveyed”. I tried to make that fit into your idea of the banana and squishy hammer, but now we’re several levels of abstraction deep using analogies to discuss analogies so it’s getting complicated to communicate clearly.

> Simon is saying don't use a banana as a hammer.

Which I agree with.

replies(1): >>45103841 #
1. tsunamifury ◴[02 Sep 25 14:48 UTC] No.45103841[source]▶
This is the type of comment that has been killing HN lately. “I agree with you but I want to disagree because I’m generally just that type of person. Also I am unable to tell my disagreeing point adds nothing.”
replies(1): >>45103970 #
2. latexr ◴[02 Sep 25 14:58 UTC] No.45103970[source]▶
Except that’s not what I’m saying at all. If anything, the “type of comment that has been killing HN” (and any community) are those who misunderstand and criticise what someone else says without providing any insight while engaging in ad hominem attacks (which are explicitly against the HN guidelines). It is profoundly ironic you are actively attacking others for the exact behaviour you are engaging in. I will kindly ask you do not do that. You are the first person in this immediate thread being rude and not adding to the collective understanding of the argument.

We are all free to agree with one part of an argument while disagreeing with another. That’s what healthy discourse is, life is not black and white. As way of example, if one says “apples are tasty because they are red”, it is perfectly congruent to agree apples are tasty but disagree that their colour is the reason. And by doing so we engage in a conversation to correct a misconception.

replies(1): >>45110655 #
3. tsunamifury ◴[02 Sep 25 23:56 UTC] No.45110655[source]▶
More of the same