How to Build a Chess Engine and Fail

I especially enjoyed the link to The Bitter Lesson by Rich Sutton, which I hadn't read before. Now I wonder what "discoveries" have been built into today's AI models and how they might come to be detrimental.

http://www.incompleteideas.net/IncIdeas/BitterLesson.html

Has there ever been a serious effort to play chess by "rule-based" methods as opposed to search?

(... other than the evaluation function being based on handwritten or learned rules)

In the past, yes. That was essentially the approach up until the 1980s. Computers were too slow to run millions of evaluations per move, so you got a lot of efforts in a more heuristic-driven direction.

Examples of heuristic engines are Hans Berliner’s CAPS-II and PARADISE by David Wilkins. PARADISE used a database of 200 rules.

There are also engines that try to retroactively apply rules to Stockfish evals but they’re fairly confusing in my experience. It’s like being told the answer and having to come up with a reason after the fact.

Probably LLM maximalist ideas that suggest infinite “scaling laws” for LLMs (they are not laws), leading to ridiculous conclusions like building a $1 trillion cluster is the fastest way to AGI. People like Leopold Aschenbrenner are in this camp.

Imagine if LLMs were the only way we had to play chess. You’d need a centralized server and peak performance wouldn't even best a grandmaster. You spend $1 trillion building a super cluster because that’s all you know.

< - - This is where AI is today.

And then some startup creates Stockfish, a chess engine better than any LLM or grandmaster and can run on a smartphone.