←back to thread

Something weird is happening with LLMs and chess

(dynomight.substack.com)
688 points crescit_eundo | 2 comments | 14 Nov 24 17:05 UTC | HN request time: 0.417s | source
Show context
lukev ◴[15 Nov 24 01:36 UTC] No.42143161[source]▶
I don't necessarily believe this for a second but I'm going to suggest it because I'm feeling spicy.

OpenAI clearly downgrades some of their APIs from their maximal theoretic capability, for the purposes of response time/alignment/efficiency/whatever.

Multiple comments in this thread also say they couldn't reproduce the results for gpt3.5-turbo-instruct.

So what if the OP just happened to test at a time, or be IP bound to an instance, where the model was not nerfed? What if 3.5 and all subsequent OpenAI models can perform at this level but it's not strategic or cost effective for OpenAI to expose that consistently?

For the record, I don't actually believe this. But given the data it's a logical possibility.

replies(3): >>42143229 #>>42143264 #>>42144445 #
zeven7 ◴[15 Nov 24 01:55 UTC] No.42143264[source]▶
Why do you doubt it? I thought it was well known that Chat GPT has degraded over time for the same model, mostly for cost saving reasons.
replies(1): >>42143324 #
permo-w ◴[15 Nov 24 02:05 UTC] No.42143324[source]▶
ChatGPT is - understandably - blatantly different in the browser compared to the app, or it was until I deleted it anyway
replies(1): >>42143446 #
lukan ◴[15 Nov 24 02:28 UTC] No.42143446[source]▶
I do not understand that. The app does not do any processing, just a UI to send text to and from the server.
replies(1): >>42144820 #
1. isaacfrond ◴[15 Nov 24 08:05 UTC] No.42144820[source]▶
There is a small difference between the app and the browser. before each session, the llm is started with a systems prompt. these are different for the app and the browser. You can find them online somewhere, but iirc the app is instructed to give shorter answers
replies(1): >>42149617 #
2. bongodongobob ◴[15 Nov 24 18:39 UTC] No.42149617[source]▶
Correct, it's different in a mobile browser too, the system prompt tells it to be brief/succinct. I always switch to desktop mode when using it on my phone.