←back to thread

Some thoughts on LLMs and software development

(martinfowler.com)
416 points floverfelt | 2 comments | 28 Aug 25 18:52 UTC | HN request time: 0.001s | source
Show context
oo0shiny ◴[28 Aug 25 22:32 UTC] No.45057794[source]▶
> My former colleague Rebecca Parsons, has been saying for a long time that hallucinations aren’t a bug of LLMs, they are a feature. Indeed they are the feature. All an LLM does is produce hallucinations, it’s just that we find some of them useful.

What a great way of framing it. I've been trying to explain this to people, but this is a succinct version of what I was stumbling to convey.

replies(5): >>45060348 #>>45060455 #>>45061299 #>>45061334 #>>45061655 #
jstrieb ◴[29 Aug 25 04:53 UTC] No.45060348[source]▶
I have been explaining this to friends and family by comparing LLMs to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better.

https://jstrieb.github.io/posts/llm-thespians/

replies(4): >>45061465 #>>45061893 #>>45061997 #>>45063090 #
red75prime ◴[29 Aug 25 09:13 UTC] No.45061893[source]▶
The analogy goes down the drain when a criterion for good performance is being objectively right. Like with Reinforcement Learning from Verifiable Rewards.
replies(3): >>45062885 #>>45064907 #>>45065336 #
1. jstrieb ◴[29 Aug 25 14:51 UTC] No.45064907[source]▶
Nobody that I'd be using this analogy with is currently using LLMs for tasks that are covered by RLVF. They're asking models for factual information about the real world (Google replacement), or to generate text (write a cover letter), not the type of outputs that are verifiable within formal systems—by definition the type of output that RLVF is intended to improve. The actor analogy is still helpful for providing intuition to non-technical people who don't know how to think about LLMs, but do use them.

Also, unless I am mistaken, RLVF changes the training to make LLMs less likely to hallucinate, but in no way does it make hallucination impossible. Under the hood, the models still work the same way (after training), and the analogy still applies, no?

replies(1): >>45066936 #
2. red75prime ◴[29 Aug 25 17:26 UTC] No.45066936[source]▶
> Under the hood, the models still work the same way (after training), and the analogy still applies, no?

Under the hood we have billions of parameters that defy any simple analogies.

Operations of a network are shaped by human data. But the structure of the network is not like the human brain. So, we have something that is human-like in some ways, but deviates from humans in ways, which are unlikely to be like anything we can observe in humans (and use as a basis for analogy).