←back to thread

Something weird is happening with LLMs and chess

(dynomight.substack.com)
688 points crescit_eundo | 1 comments | 14 Nov 24 17:05 UTC | HN request time: 0.362s | source
Show context
swiftcoder ◴[15 Nov 24 07:57 UTC] No.42144784[source]▶
I feel like the article neglects one obvious possibility: that OpenAI decided that chess was a benchmark worth "winning", special-cases chess within gpt-3.5-turbo-instruct, and then neglected to add that special-case to follow-up models since it wasn't generating sustained press coverage.
replies(8): >>42145306 #>>42145352 #>>42145619 #>>42145811 #>>42145883 #>>42146777 #>>42148148 #>>42151081 #
scott_w ◴[15 Nov 24 11:10 UTC] No.42145811[source]▶
I suspect the same thing. Rather than LLMs “learning to play chess,” they “learnt” to recognise a chess game and hand over instructions to a chess engine. If that’s the case, I don’t feel impressed at all.
replies(5): >>42146086 #>>42146152 #>>42146383 #>>42146415 #>>42156785 #
gamerDude ◴[15 Nov 24 12:43 UTC] No.42146383[source]▶
This is exactly what I feel AI needs. A manager AI that then hands off things to specialized more deterministic algorithms/machines.
replies(4): >>42146397 #>>42147292 #>>42150449 #>>42152158 #
1. waffletower ◴[15 Nov 24 20:16 UTC] No.42150449[source]▶
While deterministic components may be a left-brain default, there is no reason that such delegate services couldn't be more specialized ANN models themselves. It would most likely vastly improve performance if they were evaluated in the same memory space using tensor connectivity. In the specific case of chess, it is helpful to remember that AlphaZero utilizes ANNs as well.