←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 1 comments | 02 Mar 25 19:15 UTC | HN request time: 0s | source
Show context
krupan ◴[03 Mar 25 19:32 UTC] No.43245727[source]▶
Reading this article and then through the comments here, the overall argument I'm hearing here is that we should let the AI write the code and we should focus on reviewing it and testing it. We should work towards becoming good at specify a problem, and then validating the solution

Should we even be asking AI to write code? Shouldn't we just be building and training AI to solve these problems without writing any code at all? Replace every app with some focused, trained, and validated AI. Want to find the cheapest flights? Who cares what algorithm the AI uses to find them, just let it do that. Want to track your calorie intake, process payroll every two weeks, do your taxes, drive your car, keep airplanes from crashing into each other, encrypt your communications, predict the weather? Don't ask AI to clumsily write code to do these things. Just tell it to do them!

Isn't that the real promise of AI?

replies(1): >>43246207 #
simonw ◴[03 Mar 25 20:20 UTC] No.43246207[source]▶
I think that is a promise that is doomed to failure.

Something we have learned as a civilization over the past ~70 years is that deterministic algorithms are an incredibly powerful thing. Designing processes that have a guaranteed, reliable result for a known input is a phenomenal way to scale up solutions to all kinds of problems.

If we want AI to help us with that, the best way to do that is to have it write code.

replies(1): >>43246917 #
1. throwuxiytayq ◴[03 Mar 25 21:21 UTC] No.43246917[source]▶
AI is automating cognitive work of a human brain. There is barely anything deterministic, guaranteed, reliable or scalable about human brains. (To be honest, this should be apparent if you hired or worked with people.) If anything, being able to process these workloads without the meatware-specific deficiencies has terrifying scalability. The current wave of “““reasoning””” models demonstrate this: the LLM instantly emits a soup of tokens that could take you hours to analyze, greatly boosting the accuracy of the final answer. Expect a lot more of that, quantitatively and qualitatively.