Something weird is happening with LLMs and chess

I don't understand why educated people expect that an LLM would be able to play chess at a decent level.

It has no idea about the quality of it's data. "Act like x" prompts are no substitute for actual reasoning and deterministic computation which clearly chess requires.

One of the main purposes of running experiments of any sort is to find out if our preconceptions are accurate. Of course, if someone is not interested in that question, they might as well choose not to look through the telescope.

Sadly there’s a common sentiment on HN that testing obvious assumptions is a waste of time

Not only on HN. Trying to publish a scientific article that does not contain the word 'novel' has become almost impossible. No one is trying to reproduce anyones claims anymore.