←back to thread

Something weird is happening with LLMs and chess

(dynomight.substack.com)
688 points crescit_eundo | 1 comments | 14 Nov 24 17:05 UTC | HN request time: 0.001s | source
Show context
niobe ◴[15 Nov 24 00:40 UTC] No.42142885[source]▶
I don't understand why educated people expect that an LLM would be able to play chess at a decent level.

It has no idea about the quality of it's data. "Act like x" prompts are no substitute for actual reasoning and deterministic computation which clearly chess requires.

replies(20): >>42142963 #>>42143021 #>>42143024 #>>42143060 #>>42143136 #>>42143208 #>>42143253 #>>42143349 #>>42143949 #>>42144041 #>>42144146 #>>42144448 #>>42144487 #>>42144490 #>>42144558 #>>42144621 #>>42145171 #>>42145383 #>>42146513 #>>42147230 #
TZubiri ◴[15 Nov 24 01:45 UTC] No.42143208[source]▶
Bro, it actually did play chess, didn't you read the article?
replies(1): >>42143417 #
mandevil ◴[15 Nov 24 02:24 UTC] No.42143417[source]▶
It sorta played chess- he let it generate up to ten moves, throwing away any that weren't legal, and if no legal move was generated by the 10th try he picked a random legal move. He does not say how many times he had to provide a random move, or how many times illegal moves were generated.
replies(2): >>42143790 #>>42147803 #
TZubiri ◴[15 Nov 24 15:33 UTC] No.42147803[source]▶
Ah, I didn't see the ilegal move discarding.
replies(1): >>42147907 #
1. mandevil ◴[15 Nov 24 15:46 UTC] No.42147907[source]▶
That was for the OpenAI games- including the ones that won. For the ones he ran himself with open source LLM's he restricted their grammar to just be legal moves, so it could only respond with a legal move. But that was because of a separate process he added on top of the LLM.

Again, this isn't exactly HAL playing chess.