←back to thread

Hallucinations in code are the least dangerous form of LLM mistakes

(simonwillison.net)
371 points ulrischa | 1 comments | 02 Mar 25 19:15 UTC | HN request time: 0.207s | source
Show context
why-el ◴[03 Mar 25 03:07 UTC] No.43237886[source]▶
I am not so sure. Code by one LLM can be reviewed by another. Puppeteer like solutions will exist pretty soon. "Given this change, can you confirm this spec".

Even better, this can carry on for a few iterations. And both LLMs can be:

1. Budgeted ("don't exceed X amount")

2. Improved (another LLM can improve their prompts)

and so on. I think we are fixating on how _we_ do things, not how this new world will do their _own_ thing. That to me is the real danger.

replies(2): >>43238020 #>>43238560 #
tylerchurch ◴[03 Mar 25 03:30 UTC] No.43238020[source]▶
> Code by one LLM can be reviewed by another

Reviewed against what? Who is writing the specs?

replies(1): >>43238146 #
1. why-el ◴[03 Mar 25 03:55 UTC] No.43238146[source]▶
the user who wants it? and a premature retort: if the feedback is "the user / PM / stakeholder could be wrong", then... that's where we are. A "refiner" LLM can be fronted (Replit is playing with this for instance).

To be clear: this is not something I do currently, but my point is that one needs to detach from how _we_ engineers do this for a more accurate evaluation of whether these things truly do not work.