←back to thread

GPT-4 and professional benchmarks: the wrong answer to the wrong question

(aisnakeoil.substack.com)
340 points agomez314 | 2 comments | 21 Mar 23 13:12 UTC | HN request time: 0.401s | source
Show context
macawfish ◴[21 Mar 23 13:51 UTC] No.35246072[source]▶
I disagree that these are "the wrong questions", but I do think we need to try and be nuanced about what these kinds of results actually mean.

The potential for these tools to impact labor markets is huge, no matter what they're "actually" or "essentially" capable of.

I'm a little tired of the arguments that the large language models are just regurgitating memorized output, I think it's now clear that higher level capabilities are emerging in these models and we need to take this seriously as a social/economic/political challenge.

This is "industrial revolution" level technology.

replies(2): >>35247306 #>>35249279 #
1. tarruda ◴[21 Mar 23 15:08 UTC] No.35247306[source]▶
> I think it's now clear that higher level capabilities are emerging in these models and we need to take this seriously as a social/economic/political challenge.

It is a hard truth to face. I admit I always feel a little bit of happiness when someone shows me a stupid error ChatGPT made, as if it would somehow invalidate all the awesome things it can do and the impact it will certainly have on all of us. What does it matter if ChatGPT is conscious or not when it can clearly automate a lot of work we previously considered to be creative?.

Since last year I started to seriously take a look at AI and started learning about LLMs. Until a few days ago I hadn't bought the explanation that these things are just predicting the next word, but I accepted it once I started running the Alpaca/Llama locally on my computer.

The concept of predicting words based on statistics seems simple, but clearly complex behavior emerges from it. Maybe our own intelligence emerges from simple primitives too?

replies(1): >>35250812 #
2. cubefox ◴[21 Mar 23 18:55 UTC] No.35250812[source]▶
One possible such simple primitive is predictive coding, where the brain is hypothesized to predict experience rather than text: https://slatestarcodex.com/2017/09/05/book-review-surfing-un...