←back to thread

GPT-4 and professional benchmarks: the wrong answer to the wrong question

(aisnakeoil.substack.com)
340 points agomez314 | 5 comments | 21 Mar 23 13:12 UTC | HN request time: 0.888s | source
Show context
dcolkitt ◴[21 Mar 23 13:57 UTC] No.35246164[source]▶
GPT is a very impressive technical achievement. But that technical achievement is more in the field of compression rather than intelligence.
replies(1): >>35246563 #
sebzim4500 ◴[21 Mar 23 14:23 UTC] No.35246563[source]▶
I don't think compression and intelligence can be disentangled in this way.
replies(1): >>35247065 #
1. cjbprime ◴[21 Mar 23 14:53 UTC] No.35247065[source]▶
Agree: compression is straightforwardly prediction.

Prediction is intelligence when you're able to do it well across disparate novel tasks.

(Of course definitions are idiosyncratic, but I'm curious if anyone disagrees with these ones.)

replies(2): >>35250990 #>>35271482 #
2. cubefox ◴[21 Mar 23 19:08 UTC] No.35250990[source]▶
It seems that intelligence also is about explanation, apart from prediction. Humans not just try to predict future evidence from current evidence, or use the current evidence to confirm given hypotheses, but they also try to find an hypothesis which best explains this evidence. It's not quite clear how explanation would relate to compression.
replies(2): >>35252719 #>>35260036 #
3. kfarr ◴[21 Mar 23 21:19 UTC] No.35252719[source]▶
Explanation is decompression, using a limited set of tokens to model a more complex output
4. IIAOPSW ◴[22 Mar 23 12:31 UTC] No.35260036[source]▶
Explanation is when the number of bits in your model is smaller than the number of bits in the system. To understand is to have a compression good enough to store in your working memory.
5. Matumio ◴[23 Mar 23 07:23 UTC] No.35271482[source]▶
> Prediction is intelligence

Depends on your definition of "intelligence". The big missing part is the ability to explore, try new things, to act (enactivism). Basically to become part of the environment, instead of being a sealed box with frozen weights.

By predicting characters, the system had to master, digest, maybe even understand, all the cultural human knowledge it got in text form. Now let's aim for the process that generated this knowledge in the first place.