←back to thread

Reasoning models reason well, until they don't

(arxiv.org)
214 points optimalsolver | 2 comments | 31 Oct 25 09:23 UTC | HN request time: 0.481s | source
Show context
My_Name ◴[31 Oct 25 11:10 UTC] No.45770715[source]▶
I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it.

The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confidence.

replies(9): >>45770777 #>>45770879 #>>45771048 #>>45771093 #>>45771274 #>>45771331 #>>45771503 #>>45771840 #>>45778422 #
pimeys ◴[31 Oct 25 12:27 UTC] No.45771274[source]▶
I just got this from codex yesterday:

"I wasn’t able to finish; no changes were shipped."

And it's not the first time.

replies(2): >>45771434 #>>45771639 #
1. darkwater ◴[31 Oct 25 13:14 UTC] No.45771639[source]▶
Have you threatened it with a 2 in the next round of performance reviews?
replies(1): >>45785965 #
2. conception ◴[01 Nov 25 22:21 UTC] No.45785965[source]▶
I usually stick with “lives will be lost if you fail at this.” standard.