Something weird is happening with LLMs and chess

I feel like the article neglects one obvious possibility: that OpenAI decided that chess was a benchmark worth "winning", special-cases chess within gpt-3.5-turbo-instruct, and then neglected to add that special-case to follow-up models since it wasn't generating sustained press coverage.

Of course it's a benchmark worth winning, has been since Watson. And before that even with mechanical turks.

This seems quite likely to me, but did they special case it by reinforcement training it into the LLM (which would be extremely interesting in how they did it and what its internal representation looks like) or is it just that when you make an API call to OpenAI, the machine on the other end is not just a zillion-parameter LLM but also runs an instance of Stockfish?

That's easy to test, invent a new chess variant and see how the model does.

Both an LLM and Stockfish would fail that test.

You're imagining LLMs don't just regurgitate and recombine things they already know from things they have seen before. A new variant would not be in the dataset so would not be understood. In fact this is quite a good way to show LLMs are NOT thinking or understanding anything in the way we understand it.

To be fair, they say

> Theory 2: GPT-3.5-instruct was trained on more chess games.

I suspect the same thing. Rather than LLMs “learning to play chess,” they “learnt” to recognise a chess game and hand over instructions to a chess engine. If that’s the case, I don’t feel impressed at all.

Yes, came here to say exactly this. And it's possible this specific model is "cheating", for example by identifying a chess problem and forwarding it to a chess engine. A modern version of the Mechanical Turk.

That's the problem with closed models, we can never know what they're doing.

Yes, that's how you can really tell if the model is doing real thinking and not recombinating things. If it can correctly play a novel game, then it's doing more than that.

No LLM model is doing any thinking.

I wonder what the minimal amount of change qualifies as novel?

"Chess but white and black swap their knights" for example?

Recognize and hand over to a specialist engine? That might be useful for AI. Maybe I am missing something.

If that were the case, pumping big Llama chock full of chess games would produce good results. It didn't.

The only way it could be true is if that model recognized and replayed the answer to the game from memory.

Nobody is claiming that Stockfish is learning generalizable concepts that can one day meaningfully replace people in value creating work.

It's because this is standard practice since the early days - there's nothing newsworthy in this at all.

That's something completely different than what the OP suggests and would be a scandal if true (i.e. gpt-3.5-turbo-instruct actually using something else behind the scenes).

In both scenarios it would perform poorly on that.

If the chess specialization was done through reinforcement learning, that's not going to transfer to your new variant, any more than access to Stockfish would help it.

By that standard (and it is a good standard), none of these "AI" things are doing any thinking

How do you think AI are (correctly) solving simple mathematical questions which they have not trained for directly? They hand it over to a specialist maths engine.

How do you define thinking?

Ironically it's probably a lot closer to what a super-human AGI would look like in practice, compared to just an LLM alone.

It is and would be useful, but it would be quite a big lie to the public, but more importantly to paying customers, and even more importantly to investors.

This is exactly what I feel AI needs. A manager AI that then hands off things to specialized more deterministic algorithms/machines.

Basically what Wolfram Alpha rolled out 15 years ago.

It was impressive then, too.

TBH I think a good AI would have access to a Swiss army knife of tools and know how to use them. For example a complicated math equation, using a calculator is just smarter than doing it in your head.

We already have the chess "calculator", though. It's called stockfish. I don't know why you'd ask a dictionary how to solve a math problem.

Being fast at doing linear algebra computations. (Is there any other kind?!)

Do you have a link to the results from fine-tuning a Llama model on chess? How do they compare to the base models in the article here?

Right. To me, this is the "agency" thing, that I still feel like is somewhat missing in contemporary AI, despite all the focus on "agents".

If I tell an "agent", whether human or artificial, to win at chess, it is a good decision for that agent to decide to delegate that task to a system that is good at chess. This would be obvious to a human agent, so presumably it should be obvious to an AI as well.

This isn't useful for AI researchers, I suppose, but it's more useful as a tool.

(This may all be a good thing, as giving AIs true agency seems scary.)

People ask LLM’s to do all sorts of things they’re not good at.

The point was such a question could not be used to tell whether the llm was calling a chess engine

Why couldn't they add a tool that literally calls stockfish or a chess ai behind the scenes with function calling and buffer the request before sending it back to the endpoint output interface?

As long as you are training it to make a tool call, you can add and remove anything you want behind the inference endpoint accessible to the public, and then you can plug the answer back into the chat ai, pass it through a moderation filter, and you might get good output from it with very little latency added.

A generalist AI with a "chatty" interface that delegates to specialized modules for specific problem-solving seems like a good system to me.

"It looks like you're writing a letter" ;)

I wonder what would happen with a game that is mostly chess (or chess with truly minimal variations) but with all the names changed (pieces, moves, "check", etc, all changed). The algebraic notation is also replaced with something else so it cannot be pattern matched against the training data. Then you list the rules (which are mostly the same as chess).

None of these changes are explained to the LLM, so if it can tell it's still chess, it must deduce this on its own.

Would any LLM be able to play at a decent level?

The point of creating a service like this is for it to be useful, and if recognizing and handing off tasks to specialized agents isn't useful, i don't know what is.

You say this quite confidently, but LLMs do generalize somewhat.

Multi Agent LLM's are already a thing.

musical goalposts, gotta love it.

These LLM's just exhibited agency.

Swallow your pride.

Lets clip this in the bud before it grows wings.

If this was part of the offering: “we can recognise requests and delegate them to appropriate systems,” I’d understand and be somewhat impressed but the marketing hype is missing this out.

Most likely because they want people to think the system is better than it is for hype purposes.

I should temper my level of impressed with only if it’s doing this dynamically . Hardcoding recognition of chess moves isn’t exactly a difficult trick to pull given there’s like 3 standard formats…

If I was sold a product that can generically solve problems I’d feel a bit ripped off if I’m told after purchase that I need to build my own problem solver and way to recognise it…

If I was sold a general AI problem solving system, I’d feel ripped off if I learned that I needed to build my own problem solver and hook it up after I’d paid my money…

Ah okay, I missed that.

"Does it generalize past the training data" has been a pre-registered goalpost since before the attention transformer architecture came on the scene.

Maybe they even delegate it to a chess engine internally via the tool use and the LLM uses that.

  >'thinking' vs 'just recombinating things

If there is a difference, and LLM's can do one but not the other...

  >By that standard (and it is a good standard), none of these "AI" things are doing any thinking

  >"Does it generalize past the training data" has been a pre-registered goalpost since before the attention transformer architecture came on the scene.

Then what the fuck are they doing.

Learning is thinking, reasoning, what have you.

Move goalposts, re-define words, it won't matter.

You're speaking like it's confirmed. Do you have any proof?

Again, the comment you initially responded to was not talking about faking it by using a chess engine. You were the one introducing that theory.

Somehow they're not in the limelight, and lack a well-known open-source runner implementation (like llama.cpp).

Given the potential, they should be winning hands down; where's that?

The problem is simply that the company has not been open about how it works, so we're all just speculating here.

Fair!

So… we’re at expert systems again?

That’s how the AI winter started last time.

This is a relatively recent development (<3 months), at least for OpenAI, where the model will generate code to solve math and use the response

Chess might not be a great example, given that most people interested in analyzing chess moves probably know that chess engines exist. But it's easy to find examples where this approach would be very helpful.

If I'm an undergrad doing a math assignment and want to check an answer, I may have no idea that symbolic algebra tools exist or how to use them. But if an all-purpose LLM gets a screenshot of a math equation and knows that its best option is to pass it along to one of those tools, that's valuable to me even if it isn't valuable to a mathematician who would have just cut out of the LLM middle-man and gone straight to the solver.

There are probably a billion examples like this. I'd imagine lots of people are clueless that software exists which can help them with some problem they have, so an LLM would be helpful for discovery even if it's just acting as a pass-through.

It is good to see other people buttressing Stephen Wolfram's ego. It is extraordinarily heavy work and Stephen can't handle it all by himself.

While deterministic components may be a left-brain default, there is no reason that such delegate services couldn't be more specialized ANN models themselves. It would most likely vastly improve performance if they were evaluated in the same memory space using tensor connectivity. In the specific case of chess, it is helpful to remember that AlphaZero utilizes ANNs as well.

It looks like you have a deja vu

No, I don’t have proof and I never suggested I did. Yes, it’s 100% hypothetical but I assumed everyone engaging with me understood that.

If they came out and said it, I don’t see the problem. LLM’s aren’t the solution for a wide range of problems. They are a new tool but not everything is a nail.

I mean it already hands off a wide range of tasks to python… this would be no different.

But it already hands off plenty of stuff to things like python. How would this be any different.

They’ve been doing that a lot longer than three months. ChatGPT has been handing stuff off to python for a very long time. At least for my paid account anyway.

This is exactly it. Here’s the pull request where chess evals were added: https://github.com/openai/evals/pull/45.

Wasn't that the basis of computing and technology in general? Here is one tedious thing, let's have a specific tool that handles it instead of wasting time and efforts. The fact is that properly using the tool takes training and most of current AI marketing are hyping that you don't need that. Instead, hand over the problem to a GPT and it will "magically" solve it.

Making the OP feel threatened/emotionally attached/both enough to call the language model a rival / companion / peer instead of a tool.

Even knowing that the software exists isn't enough. You have to learn how to use the thing.

Next thing, the "manager AIs" start stack ranking the specialized "worker AIs".

And the worker AIs "evolve" to meet/exceed expectations only on tasks directly contributing to KPIs the manager AIs measure for - via the mechanism of discarding the "less fit to exceed KPIs".

And some of the worker AIs who're trained on recent/polluted internet happen to spit out prompt injection attacks that work against the manager AIs rank stacking metrics and dominate over "less fit" worker AIs. (Congratulations, we've evolved AI cancer!) These manager AIs start performing spectacularly badly compared to other non-cancerous manager AIs, and die or get killed off by the VC's paying for their datacenters.

Competing manager AIs get training, perhaps on on newer HN posts discussing this emergent behavior of worker AIs, and start to down rank any exceptionally performing worker AIs. The overall trends towards mediocrity becomes inevitable.

Some greybread writes some Perl and regexes that outcompete commercial manager AIs on pretty much every real world task, while running on a 10 year old laptop instead of a cluster of nuclear powered AI datacenters all consuming a city's worth of fresh drinking water.

Nobody in powerful positions care. Humanity dies.

Nice. Even the tiniest rule, I strongly suspect, would throw off pattern matching. “Every second move, swap the name of the piece you move to the last piece you moved.”

If you mean “uses bin/python to run Python code it wrote” then that’s a bit different to “recognises chess moves and feeds them to Stockfish.”

If a human said they could code, you don’t expect them to somehow turn into a Python interpreter and execute it in their brain. If a human said they could play chess, I’d raise an eyebrow if they just played the moves Stockfish gave them against me.

That's not much different from a compiler being rigged to recognize a specific benchmark program and spit out a canned optimization.

What is an "expert system" to you? In AI they're just series of if-then statements to encode certain rules. What non-trivial part of an LLM reaching out to a chess AI does that describe?

The initial LLM acts as an intention detection mechanism switch.

To personify LLM way too much:

It sees that a prompt of some kind wants to play chess.

Knowing this it looks at the bag of “tools” and sees a chess tool. It then generates a response which eventually causes a call to a chess AI (or just chess program, potentially) which does further processing.

The first LLM acts as a ton of if-then statements, but automatically generated (or brute-forcly discovered) through training.

You still needed discrete parts for this system. Some communication protocol, an intent detection step, a chess execution step, etc…

I don’t see how that differs from a classic expert system other than the if statement is handled by a statistical model.

You take a picture of a chess board and send it to ChatGPT and it replies with the current evaluation and the best move/strategy for black and white.

.. or a Volkswagen recognising an emissions test and turning off power mode...

Lolol. It's a chess thread, say it.

We are pawns, hoping to be maybe a Rook to the King by endgame.

Some think we can promote our pawns to Queens to match.

Luckily, the Jester muses!