(dynomight.substack.com)

688 points crescit_eundo | 1 comments | 14 Nov 24 17:05 UTC | HN request time: 0.21s | source

Show context

swiftcoder ◴[15 Nov 24 07:57 UTC] No.42144784[source]▶

I feel like the article neglects one obvious possibility: that OpenAI decided that chess was a benchmark worth "winning", special-cases chess within gpt-3.5-turbo-instruct, and then neglected to add that special-case to follow-up models since it wasn't generating sustained press coverage.

replies(8): >>42145306 #>>42145352 #>>42145619 #>>42145811 #>>42145883 #>>42146777 #>>42148148 #>>42151081 #

1. vimbtw ◴[15 Nov 24 21:08 UTC] No.42151081[source]▶

>>42144784 #

This is exactly it. Here’s the pull request where chess evals were added: https://github.com/openai/evals/pull/45.

↑

Something weird is happening with LLMs and chess