The 'scientific method' is not something that can be reduced to formal reasoning. Bayesian inference is a type of formal reasoning for making predictions, and as such, is a mathematical 'toy model' that doesn't correspond to the real universe.
Yudkowsky worships pure mathematics, but he always had it 'arse backwards'. It's not pure mathematics that's the ideal, it's numerical and iterated methods and heuristics (cognition). Pure math can only be applied to idealized situations, whereas numerical methods and heuristics apply everywhere. So in fact, it's numerical methods that are fundamental, and pure math that's the imperfect idealization!
Yudkowsky read too much Tegmark in his youth and was sucked in by the idea that 'everything is mathematics' (or 'everything is information'). But to repeat, this is all 'arse backwards'. Thank goodness that I read some Sean Carroll and debated with a friend of Sean's on his forum; that's what finally talked me out of all that Tegmark multiverse/'reality is a simulation' nonsense.
It's the physical world that's fundamental, cognition is next level of abstraction up, and pure mathematics is a top-level abstraction (it's not fundamental). As Bayesian inference (and all formal methods) are part of the domain of pure math, they can't serve as a foundation for rationality.
Cognition is more fundamental than math (because it's closer to the base level of reality - physics).
As I commented recently to Scott Aaronson on his blog, what distinguishes cognition from pure math, is that pure math is about fixed equations, whereas cognition is about heuristics , iteration and numerical methods. But in fact, P≠NP implies that cognition is the more fundamental. See:
https://www.scottaaronson.com/blog/?p=3875#comments
So for instance, AIXI (a much touted mathematical model of general intelligence), is the 'fake' (imperfect) solution, whereas a workable heuristic implementation would be the correct (perfect) one. This is the complete reverse of what Yudkowsky thinks.