←back to thread

Reasoning models reason well, until they don't

(arxiv.org)
214 points optimalsolver | 2 comments | 31 Oct 25 09:23 UTC | HN request time: 0.425s | source
Show context
WesolyKubeczek ◴[31 Oct 25 09:50 UTC] No.45770141[source]▶
It’s because they generate a seeming of reasoning, and don’t actually reason!

(Slams the door angrily)

(stomps out angrily)

(touches the grass angrily)

replies(2): >>45770177 #>>45770434 #
1. samuell ◴[31 Oct 25 09:55 UTC] No.45770177[source]▶
Yea, a bit like a cheating student rote memorizing and copying another students technique for solving a type of problem, and failing hard as soon as there's too much variation from the original problem.
replies(1): >>45770219 #
2. fsloth ◴[31 Oct 25 10:01 UTC] No.45770219[source]▶
Yes!

That said the input space of supported problems is quite large and you can configure the problem parametrs quite flexibly.

I guess the issue is that what the model _actually_ provides you is this idiot savant who has pre-memorized everything without offering a clear index that would disambiguate well-supported problems from ”too difficult” (i.e. novel) ones