←back to thread

An LLM is a lossy encyclopedia

(simonwillison.net)
509 points tosh | 8 comments | | HN request time: 1.936s | source | bottom

(the referenced HN thread starts at https://news.ycombinator.com/item?id=45060519)
1. alganet ◴[] No.45125643[source]
People don't interact with encyclopedias by chatting with them.

Also, why would I want a lossy encyclopedia? Disk space is cheaper than GPUs. I can host searchable dumps of the full Wikipedia and Stack Overflow in cheap commodity hardware.

The only good point of the analogy is that is self-declared as questionable.

replies(1): >>45125801 #
2. simonw ◴[] No.45125801[source]
"Also, why would I want a lossy encyclopedia?"

That is exactly the point of the analogy. A lossy encyclopedia is obviously a bad thing! The analogy helps illustrate why using raw LLMs to look things up in the same way as an encyclopedia is a bad idea.

replies(1): >>45137754 #
3. alganet ◴[] No.45137754[source]
I'm puzzled by this part then:

> The key thing is to develop an intuition for questions it can usefully answer vs questions that are at a level of detail where the lossiness matters.

If that intuition is not explained, it's no different than "works for me, you're using it wrong", which is definitely an argument, but not the best one.

I mean, I'm confident lots of people can use LLMs well, but I don't understand how the analogy is supposed to teach anything about it.

replies(1): >>45139646 #
4. simonw ◴[] No.45139646{3}[source]
The core analogy is intended as a warning: this is a thing that looks like an encyclopedia but really isn't.

There's not much I can do about the intuition thing. I've been trying to figure out ways to teach people to use LLMs for over three years now, but it genuinely comes down to them being utterly weird and unintuitive pieces of technology that pretend to be easy to use when they aren't.

The only way to get truly competent with them is to put in the time deliberately experimenting to figure out what does and doesn't work.

replies(1): >>45145279 #
5. alganet ◴[] No.45145279{4}[source]
Well, Claude Code is sold as an easy to use magic tool.

https://www.anthropic.com/claude-code

> Unleash Claude’s raw power directly in your terminal.

> Search million-line codebases instantly.

> Turn hours-long workflows into a single command.

> Your tools. Your workflow. Your codebase, evolving at thought speed.

--

> Watch as Claude Code tackles an unfamiliar Next.js project, builds new functionality, creates tests, and fixes what’s broken

No mention of "weird and unintuitive". It sounds like a dream. It sounds like it would accept a prompt for a niche raspberry pi skeleton and work with it. Can you really blame a beginner for buying the product and being disappointed?

You should be angry at the LLM companies. While you're trying to teach people, they're working to generate tons of unsatisfied customers. And they come here, and you answer them for free? It doesn't make much sense to me.

replies(1): >>45146802 #
6. simonw ◴[] No.45146802{5}[source]
The reason I shout "weird and unintuitive" from the rooftops is that no LLM vendor will ever describe their weird and unintuitive products that way.
replies(1): >>45148189 #
7. slacktivism123 ◴[] No.45148189{6}[source]
Describing the commercial offerings as "weird and unintuitive" is a weak criticism palatable to corporate comms teams. It suggests a fault in the user ("you're holding it wrong") rather than deficiencies inherent to LLM architecture. No amount of marketing can fix the lethal trifecta or the hallucination problem, can it?

https://www.anthropic.com/solutions/code-modernization:

    Generate dependency graphs, identify dead code, and prioritize refactoring based on code complexity metrics and business impact.
    Transform legacy codebases systematically while maintaining business continuity.
    Claude Code preserves critical business logic while modernizing to current frameworks.
    Claude Code can seamlessly create unit tests for refactored code, identify missing test coverage, and help write regression tests.
    Identify and patch vulnerabilities while maintaining regulatory compliance patterns embedded in legacy systems.
    Create modern documentation from undocumented legacy code, capturing institutional knowledge before it's lost.
replies(1): >>45148374 #
8. simonw ◴[] No.45148374{7}[source]
OpenAI actually put out an interesting paper on addressing hallucination yesterday, but I've not spent enough time with it to judge how credible it is: https://openai.com/index/why-language-models-hallucinate/

I don't particularly care how these companies market their software - what I care about is figuring out what these things can actually do and what they're genuinely useful for, then helping other people use them in as productive a way as possible given their inherent flaws.