←back to thread

Reasoning models reason well, until they don't

(arxiv.org)
214 points optimalsolver | 4 comments | 31 Oct 25 09:23 UTC | HN request time: 0s | source
Show context
brap ◴[31 Oct 25 10:27 UTC] No.45770407[source]▶
I wonder if we can get models to reason in a structured and verifiable way, like we have formal logic in math.
replies(3): >>45770419 #>>45775771 #>>45782489 #
Frieren ◴[31 Oct 25 10:28 UTC] No.45770419[source]▶
For that, you already have classical programming. It is great at formal logic math.
replies(1): >>45770489 #
1. brap ◴[31 Oct 25 10:37 UTC] No.45770489[source]▶
I think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that.

I actually believe it is technically possible, but is going to be very hard.

replies(1): >>45770979 #
2. nl ◴[31 Oct 25 11:48 UTC] No.45770979[source]▶
This is where you get the natural language tool to write the formal logic.

ChatGPT knows WebPPL really well for example.

replies(1): >>45772986 #
3. brap ◴[31 Oct 25 15:16 UTC] No.45772986[source]▶
You will need a formal language first.

Take this statement for example:

>ChatGPT knows WebPPL really well

What formal language can express this statement? What will the text be parsed into? Which transformations can you use to produce other truthful (and interesting) statements from it? Is this flexible enough to capture everything that can be expressed in English?

The closest that comes to mind is Prolog, but it doesn’t really come close.

replies(1): >>45777678 #
4. nl ◴[31 Oct 25 23:02 UTC] No.45777678{3}[source]▶
> You will need a formal language first.

No, that's the entire point!

The LLM is the bridge between natural language and a formal specification.

(WebPPL is a formal language btw. It's not unlike Prolog but is designed from the start to express lemmas probabilistically)