←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 1 comments | 02 Mar 25 19:15 UTC | HN request time: 0.204s | source
Show context
layer8 ◴[02 Mar 25 22:11 UTC] No.43235766[source]▶
> Just because code looks good and runs without errors doesn’t mean it’s actually doing the right thing. No amount of meticulous code review—or even comprehensive automated tests—will demonstrably prove that code actually does the right thing. You have to run it yourself!

I would have stated this a bit differently: No amount of running or testing can prove the code correct. You actually have to reason through it. Running/testing is merely a sanity/spot check of your reasoning.

replies(4): >>43235828 #>>43235856 #>>43236195 #>>43236756 #
dmos62 ◴[02 Mar 25 22:18 UTC] No.43235828[source]▶
Well, what if you run a complete test suite?
replies(5): >>43236019 #>>43236118 #>>43236949 #>>43237000 #>>43237496 #
layer8 ◴[02 Mar 25 22:39 UTC] No.43236019[source]▶
There is no complete test suite, unless your code is purely functional and has a small-ish finite input domain.
replies(2): >>43236080 #>>43236539 #
1. suzzer99 ◴[02 Mar 25 22:47 UTC] No.43236080[source]▶
And even then, your code could pass all tests but be a spaghetti mess that will be impossible to maintain and add features to.