←back to thread

Something weird is happening with LLMs and chess

(dynomight.substack.com)
688 points crescit_eundo | 3 comments | 14 Nov 24 17:05 UTC | HN request time: 0.503s | source
Show context
lukev ◴[15 Nov 24 01:36 UTC] No.42143161[source]▶
I don't necessarily believe this for a second but I'm going to suggest it because I'm feeling spicy.

OpenAI clearly downgrades some of their APIs from their maximal theoretic capability, for the purposes of response time/alignment/efficiency/whatever.

Multiple comments in this thread also say they couldn't reproduce the results for gpt3.5-turbo-instruct.

So what if the OP just happened to test at a time, or be IP bound to an instance, where the model was not nerfed? What if 3.5 and all subsequent OpenAI models can perform at this level but it's not strategic or cost effective for OpenAI to expose that consistently?

For the record, I don't actually believe this. But given the data it's a logical possibility.

replies(3): >>42143229 #>>42143264 #>>42144445 #
1. com2kid ◴[15 Nov 24 06:44 UTC] No.42144445[source]▶
> OpenAI clearly downgrades some of their APIs from their maximal theoretic capability, for the purposes of response time/alignment/efficiency/whatever.

When ChatGPT3.5 first came out, people were using it to simulate entire Linux system installs, and even browsing a simulated Internet.

Cool use cases like that aren't even discussed anymore.

I still wonder what sort of magic OpenAI had and then locked up away from the world in the name of cost savings.

Same thing with GPT 4 vs 4o, 4o is obviously worse in some ways, but after the initial release (when a bunch of people mentioned this), the issue has just been collectively ignored.

replies(2): >>42144529 #>>42146045 #
2. golol ◴[15 Nov 24 07:00 UTC] No.42144529[source]▶
You can still do this. People just lost interest in this stuff because it became clear to ehich degree the simulation is really being done (shallow).

Yet I do wish we had access to less finetuned/distilled/RLHF'd models.

3. ipsum2 ◴[15 Nov 24 11:44 UTC] No.42146045[source]▶
People are doing this all the time with Claude 3.5.