←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 3 comments | 02 Mar 25 19:15 UTC | HN request time: 0s | source
Show context
tombert ◴[02 Mar 25 22:02 UTC] No.43235669[source]▶
I use ChatGPT to generate code a lot, and it's certainly useful, but it has given me issues that are not obvious.

For example, I had it generate some C code to be used with ZeroMQ a few months ago. The code looked absolutely fine, and it mostly worked fine, but it made a mistake with its memory allocation stuff that caused it to segfault sometimes, and corrupt memory other times.

Fortunately, this was such a small project and I already know how to write code, so it wasn't too hard for me to find and fix, though I am slightly concerned that some people are copypasting large swaths of code from ChatGPT that looks mostly fine but hides subtle bugs.

replies(2): >>43235969 #>>43238803 #
1. zahlman ◴[03 Mar 25 06:07 UTC] No.43238803[source]▶
>though I am slightly concerned that some people are copypasting large swaths of code from ChatGPT that looks mostly fine but hides subtle bugs.

They used to do the same with Stack Overflow. But now it's more dangerous, because the code can be "subtly wrong in ways the user can't fathom" to order.

replies(2): >>43244928 #>>43245641 #
2. tombert ◴[03 Mar 25 18:21 UTC] No.43244928[source]▶
Yeah, there's effectively no limit to how much code that you can have.

We're all guilty of copypasting from Stack Overflow, but as you said, that's not made to order. In order to use the code copied from there, you will likely have to edit it, at least a bit to fit your application, meaning that it does require a bit of understanding of what you're doing.

Since ChatGPT can be completely tuned to what you want without writing code, it's far more tempting to just copy and paste from it without auditing it.

3. krupan ◴[03 Mar 25 19:23 UTC] No.43245641[source]▶
The beauty of stack overflow is that the code you are copying and pasting has been reviewed and voted on by a decent number of other programmers