Most active commenters
  • squidbeak(6)
  • darkwater(4)

←back to thread

Reasoning models reason well, until they don't

(arxiv.org)
214 points optimalsolver | 13 comments | 31 Oct 25 09:23 UTC | HN request time: 0s | source | bottom
Show context
My_Name ◴[31 Oct 25 11:10 UTC] No.45770715[source]▶
I find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it.

The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confidence.

replies(9): >>45770777 #>>45770879 #>>45771048 #>>45771093 #>>45771274 #>>45771331 #>>45771503 #>>45771840 #>>45778422 #
ftalbot ◴[31 Oct 25 11:18 UTC] No.45770777[source]▶
Every token in a response has an element of randomness to it. This means they’re non-deterministic. Even if you set up something within their training data there is some chance that you could get a nonsense, opposite, and/or dangerous result. The chance of that may be low because of things being set up for it to review its result, but there is no way to make a non-deterministic answer fully bound to solving or reasoning anything assuredly, given enough iterations. It is designed to be imperfect.
replies(4): >>45770905 #>>45771745 #>>45774081 #>>45775980 #
yuvalr1 ◴[31 Oct 25 11:39 UTC] No.45770905[source]▶
You are making a wrong leap from non-deterministic process to uncontrollable result. Most of the parallel algorithms are non-deterministic. There might be no guarantee about the order of calculation or even sometimes the final absolute result. However, even when producing different final results, the algorithm can still guarantee characteristics about the result.

The hard problem then is not to eliminate non-deterministic behavior, but find a way to control it so that it produces what you want.

replies(1): >>45771058 #
1. flavaflav2 ◴[31 Oct 25 11:58 UTC] No.45771058[source]▶
Life and a lot in our universe is non-deterministic. Some people assume science and mathematics are some universal truths rather than imperfect agreed upon understandings. Similarly many assume humans can be controlled through laws, penalties, prisons, propaganda, coercion, etc. But terrible things happen. Yes, if you set up the gutter-rails in your bowling lane, you can control the bowling ball unless it is thrown over those rails or in a completely different direction, but those rails are wide with LLMs by default, and the system instructions provided it aren’t rules, they are an inherently faulty way to coerce a non-deterministic system. But, yes, if there’s absolutely no way to do something, and you’re aware of every possible way a response or tool could affect things, and you have taken every possible precaution, you can make it behave. That’s not how people are using it though, and we cannot control our tendency to trust that which seems trustworthy even if we are told these things.
replies(1): >>45771126 #
2. squidbeak ◴[31 Oct 25 12:08 UTC] No.45771126[source]▶
No, Science is a means of searching for those truths - definitely not some 'agreed upon understanding'. It's backed up by experimentation and reproducible proofs. You also make a huge bogus leap from science to humanities.
replies(2): >>45771371 #>>45771622 #
3. iq176 ◴[31 Oct 25 12:39 UTC] No.45771371[source]▶
Scientific method is the process. Science itself includes the study and compendium of understandings, based on a belief system that includes shared understandings just like mathematics. The foundation of these are philosophical beliefs that we can know and understand these things. For example, on a metaphysical level, if the world around us were a simulation, then science could provide understandings about that simulated universe, but not about that which is simulating it.
replies(1): >>45771733 #
4. darkwater ◴[31 Oct 25 13:12 UTC] No.45771622[source]▶
But those are still approximations to the actual underlying reality. Because the other option (and yes, it's a dichotomy) is that we already defined and understood every detail of the physics that applies to our universe.
replies(1): >>45771708 #
5. squidbeak ◴[31 Oct 25 13:21 UTC] No.45771708{3}[source]▶
Indeed, that is a dichotomy: a false one. Science is exact without being finished.
replies(1): >>45772038 #
6. squidbeak ◴[31 Oct 25 13:23 UTC] No.45771733{3}[source]▶
This I'm afraid is rubbish. Scientific proofs categorically don't depend on philosophical beliefs. Reality is measurable and the properties measured don't care about philosophy.
replies(1): >>45772324 #
7. darkwater ◴[31 Oct 25 13:56 UTC] No.45772038{4}[source]▶
So, was Newtonian physics exact already?
replies(1): >>45772146 #
8. squidbeak ◴[31 Oct 25 14:07 UTC] No.45772146{5}[source]▶
> Science is exact without being finished
replies(1): >>45772311 #
9. darkwater ◴[31 Oct 25 14:24 UTC] No.45772311{6}[source]▶
Being exact doesn't mean it is not an approximation, which was the initial topic. Being exact in science means that 2+2=4 and that can be demonstrated following a logical chain. But that doesn't make our knowledge of the universe exact. It is still an approximation. What it can be "exact" is how we obtain and reproduce the current knowledge we have of it.
replies(1): >>45774277 #
10. weltensturm ◴[31 Oct 25 14:26 UTC] No.45772324{4}[source]▶
> Reality is measurable

Heisenberg would disagree.

replies(1): >>45774272 #
11. squidbeak ◴[31 Oct 25 17:06 UTC] No.45774272{5}[source]▶
Are you arguing that the uncertainty principle derives from philosophy rather than math?
12. squidbeak ◴[31 Oct 25 17:07 UTC] No.45774277{7}[source]▶
The speed of light, or plank's constant - are these approximations?
replies(1): >>45780008 #
13. darkwater ◴[01 Nov 25 08:10 UTC] No.45780008{8}[source]▶
To our current knowledge, no. But maybe we are missing something, we cannot know. Did infrared light or ultrasound start to exist only when we realized there are things our senses cannot feel?