←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 2 comments | 02 Mar 25 19:15 UTC | HN request time: 0.539s | source
Show context
layer8 ◴[02 Mar 25 22:11 UTC] No.43235766[source]▶
> Just because code looks good and runs without errors doesn’t mean it’s actually doing the right thing. No amount of meticulous code review—or even comprehensive automated tests—will demonstrably prove that code actually does the right thing. You have to run it yourself!

I would have stated this a bit differently: No amount of running or testing can prove the code correct. You actually have to reason through it. Running/testing is merely a sanity/spot check of your reasoning.

replies(4): >>43235828 #>>43235856 #>>43236195 #>>43236756 #
nnnnico ◴[02 Mar 25 22:21 UTC] No.43235856[source]▶
not sure that human reasoning actually beats testing when checking for correctness
replies(4): >>43235988 #>>43236031 #>>43237608 #>>43239891 #
1. fragmede ◴[03 Mar 25 02:19 UTC] No.43237608[source]▶
Human reason is fine, the problem is that human attention spans aren't great at checking for correctness. I want every corner case regression tested automatically because there's always going to be some weird configuration that a human's going to forget to regression test.
replies(1): >>43240867 #
2. sarchertech ◴[03 Mar 25 11:52 UTC] No.43240867[source]▶
With any non trivial system you can’t actually test every corner case. You depend on human reason to identify the ones most likely to cause problems.