←back to thread

LLM Hallucinations in Practical Code Generation

(dl.acm.org)
65 points appwiz | 2 comments | 23 Jun 25 07:14 UTC | HN request time: 0.404s | source
Show context
simonw ◴[26 Jun 25 02:22 UTC] No.44383691[source]▶
I still don't think hallucinations in generated code matter very much. They show up the moment you try to run the code, and with the current batch of "coding agent" systems it's the LLM itself that spots the error when it attempts to run the code.

I was surprised that this paper talked more about RAG solutions than tool-use based solutions. Those seem to me like a proven solution at this point.

replies(4): >>44384474 #>>44384576 #>>44387027 #>>44388124 #
1. DarkNova6 ◴[26 Jun 25 15:02 UTC] No.44388124[source]▶
"I still don't think hallucinations in generated code matter very much"

Tell these our Python developers who don't test anything outside of a narrow happy path.

replies(1): >>44388315 #
2. simonw ◴[26 Jun 25 15:24 UTC] No.44388315[source]▶
Yeah, if you use LLMs to write code and don't test outside the happy path you're vibe coding, you're not software engineering.