←back to thread

LLM Hallucinations in Practical Code Generation

(dl.acm.org)
65 points appwiz | 1 comments | 23 Jun 25 07:14 UTC | HN request time: 0.303s | source
Show context
simonw ◴[26 Jun 25 02:22 UTC] No.44383691[source]▶
I still don't think hallucinations in generated code matter very much. They show up the moment you try to run the code, and with the current batch of "coding agent" systems it's the LLM itself that spots the error when it attempts to run the code.

I was surprised that this paper talked more about RAG solutions than tool-use based solutions. Those seem to me like a proven solution at this point.

replies(4): >>44384474 #>>44384576 #>>44387027 #>>44388124 #
1. fulafel ◴[26 Jun 25 05:59 UTC] No.44384576[source]▶
Hopefully this will be a catalyst to bring language integraed machine readable schema checking into wider use (ala Clojure), as static typing is crap for structured data/API stuff.