←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 2 comments | 02 Mar 25 19:15 UTC | HN request time: 0.458s | source
Show context
layer8 ◴[02 Mar 25 22:11 UTC] No.43235766[source]▶
> Just because code looks good and runs without errors doesn’t mean it’s actually doing the right thing. No amount of meticulous code review—or even comprehensive automated tests—will demonstrably prove that code actually does the right thing. You have to run it yourself!

I would have stated this a bit differently: No amount of running or testing can prove the code correct. You actually have to reason through it. Running/testing is merely a sanity/spot check of your reasoning.

replies(4): >>43235828 #>>43235856 #>>43236195 #>>43236756 #
1. Snuggly73 ◴[02 Mar 25 23:01 UTC] No.43236195[source]▶
Agree - case in point - dealing with race conditions. You have to reason thru the code.
replies(1): >>43239453 #
2. wfn ◴[03 Mar 25 08:12 UTC] No.43239453[source]▶
> case in point - dealing with race conditions.

100%. Case in point for case in point - I was just scratching my head over some Claude-produced lines for me, thinking if I should ask what this kind entity had in mind when using specific compiler builtins (vs. <stdatomic.h>), like, "is there logic to your madness..." :D

  size_t unique_ips = __atomic_load_n(&((ip_database_t*)arg)->unique_ip_count, __ATOMIC_SEQ_CST);
I think it just likes compiler builtins because I mentioned GCC at some point...