←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 2 comments | 02 Mar 25 19:15 UTC | HN request time: 0s | source
Show context
tombert ◴[02 Mar 25 22:02 UTC] No.43235669[source]▶
I use ChatGPT to generate code a lot, and it's certainly useful, but it has given me issues that are not obvious.

For example, I had it generate some C code to be used with ZeroMQ a few months ago. The code looked absolutely fine, and it mostly worked fine, but it made a mistake with its memory allocation stuff that caused it to segfault sometimes, and corrupt memory other times.

Fortunately, this was such a small project and I already know how to write code, so it wasn't too hard for me to find and fix, though I am slightly concerned that some people are copypasting large swaths of code from ChatGPT that looks mostly fine but hides subtle bugs.

replies(2): >>43235969 #>>43238803 #
KoolKat23 ◴[02 Mar 25 22:32 UTC] No.43235969[source]▶
And subtle bugs existed pre-2022, how often my apps are updated for "minor bug fixes" would mean this is par for the course.
replies(1): >>43236187 #
1. tombert ◴[02 Mar 25 22:59 UTC] No.43236187[source]▶
Sure, it's possible that the code it gave me was based on some incorrectly written code it scraped from Gitlab or something.

I'm not a luddite, I'm perfectly fine with people using AI for writing code. The only thing that really concerns me is that it has the potential to generate a ton of shitty code that doesn't look shitty, creating a lot of surface area for debugging.

Prior to AI, the quantity of crappy code that could be generated was basically limited by the speed in which a human could write it, but now there's really no limit.

Again, just to reiterate, this isn't "old man yells at cloud". I think AI is pretty cool, I use it all the time, I don't even have a problem with people generating large quantities of code, it's just something we have to be a bit more weary of.

replies(1): >>43241621 #
2. KoolKat23 ◴[03 Mar 25 13:43 UTC] No.43241621[source]▶
Agree, just means less time developing and more time on quality control.