←back to thread

Getting 50% (SoTA) on Arc-AGI with GPT-4o

(redwoodresearch.substack.com)
394 points tomduncalf | 1 comments | 17 Jun 24 21:51 UTC | HN request time: 0s | source
Show context
atleastoptimal ◴[18 Jun 24 04:44 UTC] No.40714152[source]▶
I'll say what a lot of people seem to be denying. GPT-4 is an AGI, just a very bad one. Even GPT-1 was an AGI. There isn't a hard boundary between non AGI and AGI. A lot of people wish there was so they imagine absolutes regarding LLM's like "they cannot create anything new" or something like that. Just think: we consider humans a general intelligence, but obviously wouldn't consider an embryo or infant a general intelligence. So at what point does a human go from not generally intelligent to generally intelligent? And I don't mean an age or brain size, I mean suite of testable abilities.

Intelligence is an ability that is naturally gradual and emerges over many domains. It is a collection of tools via which general abstractive principles can be applied, not a singular universally applicable ability to think in abstractions. GPT-4, compared to a human, is a very very small brain trained for the single purpose of textual thinking with some image capabilities. Claiming that ARC is the absolute market of general intelligence fails to account for the big picture of what intelligence is.

replies(7): >>40714189 #>>40714191 #>>40714565 #>>40715248 #>>40715346 #>>40715384 #>>40716518 #
theptip ◴[18 Jun 24 04:56 UTC] No.40714191[source]▶
This seems to be so broad a definition as to no longer mean anything useful.

People in general are interested in capabilities or economic impact, and GPT-2 cleared no notable thresholds in those regards.

I prefer the exact opposite approach: let’s use a strict definition, and have levels to make it really explicit what we are talking about.

Here is a good one:

“Levels of AGI for Operationalizing Progress on the Path to AGI”

https://arxiv.org/abs/2311.02462

replies(1): >>40714948 #
1. sigmoid10 ◴[18 Jun 24 07:15 UTC] No.40714948[source]▶
People will never agree on this. We've known about the concept of intelligence for much, much longer than computers have been around and we still don't have a common definition or set of rules to check. That also makes things like consciousness and death pretty hard to define in medicine, leading to inconsistent rules across jurisdictions. For AGI in particular, I guarantee you that no matter which test gets beaten, the majority of humanity will always just move the goalpost and claim it's not "real" AGI because "reasons." Because the opposite would mean they have to admit that they are now the lesser intelligence on the planet.