←back to thread

Some thoughts on LLMs and software development

(martinfowler.com)
416 points floverfelt | 1 comments | 28 Aug 25 18:52 UTC | HN request time: 0.215s | source
Show context
bko ◴[28 Aug 25 19:47 UTC] No.45056268[source]▶
> Certainly if we ever ask a hallucination engine for a numeric answer, we should ask it at least three times, so we get some sense of the variation.

This works on people as well!

Cops do this when interrogating. You tell the same story three times, sometimes backwards. It's hard to keep track of everything if you're lying or you don't recall clearly so you can get a sense of confidence. Also works on interviews, ask them to explain a subject in three different ways to see if they truly understand.

replies(4): >>45056315 #>>45056413 #>>45058249 #>>45061677 #
1. Terr_ ◴[28 Aug 25 20:02 UTC] No.45056413[source]▶
> This works on people as well!

Only within certain conditions or thresholds that we're still figuring out. There are many cases where the more someone recalls and communicates their memory, the more details get corrupted.

> Cops do this when interrogating.

Sometimes that's not to "get sense of the variation" but to deliberately encourage a contradiction to pounce upon it. Ask me my own birthday enough times in enough ways and formats, and eventually I'll say something incorrect.

Care must also be taken to ensure that the questioner doesn't change the details, such as by encouraging (or sometimes forcing) the witness/suspect to imagine things which didn't happen.