DeepSeek: Advancing theorem proving in LLMs through large-scale synthetic data

"Suppose you try to construct a coherent, ordered, natural world with no resource other than repeated exposure to things, and the formation of certain associative bonds. Oh, please!"

This is prof. Robinson on Kantian philosophy - check out Oxford podcasts by the way - and this quote is meant to imply that building a coherent world out of raw sensory data and statistics alone is completely and utterly impractical if not outright impossible. While I don't think he meant to refer to any kind of AI, in my mind this description also aptly describes the general method of DL neural networks. Repeated exposure to find correlation.

How does one find order through associativity alone? With AI this is not an academic problem anymore. This has become practical. Kant says it is impossible, not just unlikely.

The Kantian project and the various core issues it tries to address seems readily applicable to AI research yet I see very little mention of it. Perhaps I am just dumb though. Building a mind capable of taming tremendous sensory flux needs to, at the very least, take note of the (many) fundamental issues he raised. Issues I feel are not at all trivial to set aside. I feel we are stuck in Hume's empiricist reasoning and have yet to graduate to Kant and beyond.

Are we now somehow convinced yet again that causality and reasoning will, in fact, after all spontaneously emerge out of pure chaos? Didn't we settle the impossibility of this a few hundred years ago?

Pearl's ladder of causation (in maths and AI research) already tells you that causation cannot only be inferred from data alone. If you only have data, then you cannot generally go beyond correlations. This can be solved either by humans hardcoding their knowledge into a model that extracts causation based on that knowledge, of by giving a machine the ability to perform their own experiments. This is because there are some correlations (A - B) that are generated by causation (A -> B) whose direction of the arrow cannot be decided with data alone.

That's a significant observation by professor Pearl and I don't wish to belittle his contributions, which are significant, but I do wish to stress that it is the 18th century we need to thank for "causation cannot only be inferred from data alone" because I feel we are, to our detriment, unfairly dismissive of past thinkers.

I don't think we are being dismissive, it's just that a mathematical theory is more convincing than a philosophical one in terms of what can and cannot be achieved with data.