Reasoning models reason well, until they don't

I wonder if we can get models to reason in a structured and verifiable way, like we have formal logic in math.

For that, you already have classical programming. It is great at formal logic math.

I think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that.

I actually believe it is technically possible, but is going to be very hard.

This is where you get the natural language tool to write the formal logic.

ChatGPT knows WebPPL really well for example.

You will need a formal language first.

Take this statement for example:

>ChatGPT knows WebPPL really well

What formal language can express this statement? What will the text be parsed into? Which transformations can you use to produce other truthful (and interesting) statements from it? Is this flexible enough to capture everything that can be expressed in English?

The closest that comes to mind is Prolog, but it doesn’t really come close.

It's doing so already. All code executed on a computer, especially neural networks w/o any loops are simply doing boolean arithmetic. In fact, the computer can't do anything else other than boolean arithmetic.

> You will need a formal language first.

No, that's the entire point!

The LLM is the bridge between natural language and a formal specification.

(WebPPL is a formal language btw. It's not unlike Prolog but is designed from the start to express lemmas probabilistically)

You can get a model to write lean or something, but formal logic, while verifiable is not useful for everyday life since it's mostly incomplete and does not even take into account inductive logic.