←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 1 comments | 02 Mar 25 19:15 UTC | HN request time: 0.206s | source
Show context
burningion ◴[03 Mar 25 15:45 UTC] No.43242840[source]▶
I think another category of error that Simon skips over that breaks this argument entirely: the hallucination where the model forgets a feature.

Rather than the positive (code compiles), the negative (forgets about a core feature), can be extremely difficult to tell. Worse still, the feature can slightly drift, based upon code that's expected to be outside of the dialogue / context window.

I've had multiple times where the model completely forgot about features in my original piece of code, after it makes a modification. I didn't notice these missing / subtle changes until much later.

replies(1): >>43243011 #
1. simonw ◴[03 Mar 25 16:00 UTC] No.43243011[source]▶
That doesn't fit the definition of "hallucination" I was using here, which is a model inventing something that doesn't exist. Definitely a problem to watch out for though - I've had to remind the models to use existing functions before. I see that as an inevitable part of the back and forth with the model while iterating on code.