Something weird is happening with LLMs and chess

One of the main purposes of running experiments of any sort is to find out if our preconceptions are accurate. Of course, if someone is not interested in that question, they might as well choose not to look through the telescope.

Sadly there’s a common sentiment on HN that testing obvious assumptions is a waste of time

Not only on HN. Trying to publish a scientific article that does not contain the word 'novel' has become almost impossible. No one is trying to reproduce anyones claims anymore.

Do you think this bias is part of the replication crisis in science?

I don't think this is about replication, but even just about the initial test in the first place. In science we do often test obvious things. For example, I was a theoretical quantum physicist, and a lot of the time I knew that what I am working on will definitely work, since the maths checks out. In some sense that makes it kinda obvious, but we test it anyway.

The issue is that even that kinda obviousness is criticised here. People get mad at the idea of doing experiments when we already expect a result.