←back to thread

An LLM is a lossy encyclopedia

(simonwillison.net)
509 points tosh | 1 comments | 29 Aug 25 09:40 UTC | HN request time: 0.213s | source

(the referenced HN thread starts at https://news.ycombinator.com/item?id=45060519)
Show context
baq ◴[02 Sep 25 09:43 UTC] No.45100866[source]▶
It's also important to say what it isn't. LLM detractors, for lack of a better word, expect an oracle and then when they find out it's just a lossily compressed blob of human knowledge with natural language as a query interface they say the tool is useless.

I've got my opinion on whether that's useful or not and it's quite a bit more nuanced. You don't zoom-enhance JPEGs for a reason either.

replies(5): >>45100993 #>>45101419 #>>45101557 #>>45101734 #>>45102386 #
littlestymaar ◴[02 Sep 25 10:01 UTC] No.45100993[source]▶
> You don't zoom-enhance JPEGs for a reason either.

Tell that to the Google Pixel product team:

https://mk.absturztau.be/notes/ac4jnvjpskjc02lq

replies(1): >>45101046 #
1. baq ◴[02 Sep 25 10:09 UTC] No.45101046[source]▶
That's both perfect for this thread and terrible in general.

If you can see the analogy between text and pictures, it drives the point exactly the right way: in both cases you expect a database to know things it either can't or forgot. If it had a good picture of the zoomed in background it could probably generate a very good representation of what the cropped part would look like; same thing works with text.