Something weird is happening with LLMs and chess

I would be interested to know if the good result is repeatable. We had a similar result with a quirky chat interface in that one run gave great results (and we kept the video) but then we couldn't do it again. The cynical among us think there was a mechanical turk involved in our good run. The economics of venture capital means that there is enormous pressure to justify techniques that we think of as "cheating". And of course the companies involved have the resources.

It's repeatable. OpenAI isn't cheating.

Source: I'm at OpenAI and I was one of the first people to ever play chess against the GPT-4 base model. You may or may not trust OpenAI, but we're just a group of people trying earnestly to build cool stuff. I've never seen any inkling of an attempt to cheat evals or cheat customers.