←back to thread

Something weird is happening with LLMs and chess

(dynomight.substack.com)
688 points crescit_eundo | 1 comments | 14 Nov 24 17:05 UTC | HN request time: 0s | source
Show context
PaulHoule ◴[14 Nov 24 21:56 UTC] No.42141647[source]▶
Maybe that one which plays chess well is calling out to a real chess engine.
replies(6): >>42141726 #>>42141959 #>>42142323 #>>42142342 #>>42143067 #>>42143188 #
1. selcuka ◴[15 Nov 24 01:18 UTC] No.42143067[source]▶
I think that's the most plausible theory that would explain the sudden hike from gpt-3.5-turbo to gpt-3.5-turbo-instruct, and again the sudden regression in gpt-4*.

OpenAI also seem to augment the LLM with some type of VM or a Python interpreter. Maybe they run a simple chess engine such as Sunfish [1] which is around 1900-2000 ELO [2]?

[1] https://github.com/thomasahle/sunfish

[2] https://lichess.org/@/sunfish-engine