←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 1 comments | 02 Mar 25 19:15 UTC | HN request time: 0.276s | source
Show context
layer8 ◴[02 Mar 25 22:11 UTC] No.43235766[source]▶
> Just because code looks good and runs without errors doesn’t mean it’s actually doing the right thing. No amount of meticulous code review—or even comprehensive automated tests—will demonstrably prove that code actually does the right thing. You have to run it yourself!

I would have stated this a bit differently: No amount of running or testing can prove the code correct. You actually have to reason through it. Running/testing is merely a sanity/spot check of your reasoning.

replies(4): >>43235828 #>>43235856 #>>43236195 #>>43236756 #
nnnnico ◴[02 Mar 25 22:21 UTC] No.43235856[source]▶
not sure that human reasoning actually beats testing when checking for correctness
replies(4): >>43235988 #>>43236031 #>>43237608 #>>43239891 #
ljm ◴[02 Mar 25 22:35 UTC] No.43235988[source]▶
The production of such tests presumably requires an element of human reasoning.

The requirements have to come from somewhere, after all.

replies(1): >>43241552 #
1. MrMcCall ◴[03 Mar 25 13:34 UTC] No.43241552[source]▶
I would argue that designing and implementing a working project requires human reasoning, too, but that line of thinking seems to be falling out of fashion in favor of "best next token" guessing engines.

I know what Spock would say about this approach, and I'm with him.