Getting 50% (SoTA) on Arc-AGI with GPT-4o

> GPT-4 is an AGI, just a very bad one.

Then stop selling it as a tool to replace humans. A fast moving car breaking through a barrier and flying off the cliff could be called "an airborne means of transportation, just a very bad one" yet nobody is suggesting it should replace school busses if only we could add longer wings to it. What the LLM community refuses to see is that there is a limit to the patience and the financing the rest of the world will grant you before you're told, "it doesn't work mate."

> So at what point does a human go from not generally intelligent to generally intelligent?

Developmental psychology would be a good place to start looking for answers to this question. Also, forgetting scientific approach and going with common sense, we do not allow young humans to operate complex machinery, decide who is allowed to become a doctor, or go to jail. Intelligence is something that is not equally distributed across the human population and some of us never have much of it, yet we function and have a role in society. Our behaviour, choices, preferences, opinions are not just based on our intelligence, but often on our past experiences and circumstances. It is also not the sole quality we use to compare ourselves against each other. A not very intelligent person is capable of making the right choices (an otherwise obedient soldier refusing to press the button and blow up a building full of children); similarly, a highly intelligent person can become a hard-to-find serial criminal (a gynecologist impregnating his patients).

What intelligent and creative people hold against LLMs is not that they replace them, but that they replace them with a shit version of them relegating thousands of years of human progress and creativity to the dustbin of the models and layers of tweaks to the output that still produce unreliable crap. I think the person who wrote this sign summed it up best https://x.com/gvanrossum/status/1802378022361911711

In response to the sign: then learn to code or make art that is better than AI art.

It's an existential complaint. "Why won't the nerds make something for meeeee." Do it yourself. Make that robot.

Sucks to think that you're not that special. Most art isn't. Most music isn't. Any honest artist will agree. Most professional artists are graphic designers, not brilliant once in a generation visionaries. It's the new excuse for starving artists. AI or no, they'd still be unsuccessful. That's the way it's always been.

While that is 100% true, the real problem is that the risk of finding out whether you can make special art has significantly increased. Previously if you didn’t make it as an artist, you could still earn money with other art related tasks - in graphics, many became illustrators. In music, people made music for ads.

That plan B is now going away, and a music career will be much more like a sports career: either you make it in football, or you need to find another career where your football skills won’t be very useful.

That is obviously scary for many.

Artists who make it usually have a legend, a story to tell or be told by their friends, associates, agents, publishers, gallerists, etc. That story has a human dimension that touches the rest of us and we somehow connect to it. Van Gogh cut of his ear, we still keep talking about it and wondering why? There is nothing AI can tell us about itself, its "art". The artistic struggle with AI is not about expressing your vision on a canvas in a way that makes others feel what you want them to feel but about forcing it to generate something it is incapable of generating or programmed not to generate. We got to the point where we are given crayons programmed to not draw the things others do not want them to draw or to draw HR-approved version of what the artist wants to draw. The future is now and it's shit.

> What the LLM community refuses to see is that there is a limit to the patience and the financing the rest of the world will grant you before you're told, "it doesn't work mate."

The point about LLM's is they may have a lot of drawbacks right now but they're improving at a rapid pace. They already are very useful. There are hundreds of stories coming out of companies effectively leveraging them to replace workers in many natural-language related tasks. They're far more useful than a car that goes off a cliff.

Nobody more useful than an LLM is being effectively replaced by an LLM. Those few companies that jump the gun too early are suffering for it.

>That sign

We already have dishwashers and washing machines. Companies are working on making humanoid robots that can do those things, it's just that it's harder to develop a fully-fledged embodied humanoid than it is to create the diffusion models and LLM's being used today. It's not some conspiracy to let AI do all the fun stuff first.

Nobody is preventing anyone from making art or writing poetry. If someone finds value in AI art or writing, either you have to accept that they weren't the audience member you wanted, or you have to accept that your ability to be creative is a learnable algorithm same as anything else.

The sign reads (paraphrased): “I want AI to do my dishes so I can do art. Not do art so I can do my dishes”

Your response is “learn to art” “The nerds dont owe you anything.” “Most of You would be unsuccessful anyway”

You brought in absolutely unrelated items.

1) Learn art - that is baked into what the Sign is saying. There is no Terminal Point for being an artist.

2) Nerds dont… - Where nerds come in as a class for this conversation?

2.1) if you can speak for all nerds, please note that I sure as heck dont want Warhammer 40k, I want Star Trek.

3) Most would be unsuccessful - so what?

Are they happy practicing their craft? Do they have the choice to spend their time on those pursuits and enrich their lives, and share their joys with others around them?

Using van Gogh as an example of “artists who make it” is insane.

Which I guess is appropriate, because he was literally crazy. He suffered from psychotic episodes and delusions and died from suicide, depressed and in poverty.

That’s the opposite of “making it”. It’s zero consolation that people like his work now, he never even knew.

> Using van Gogh as an example of “artists who make it” is insane.

So is building a tool that will only generate "approved" art. We need to be able to express our idea, feelings, our perception of the world in ways that do not fit corporate standards of text, audio, or visual communication. It's part of being human.

> So is building a tool that will only generate "approved" art.

And so is eating ice cream with your forehead. Are we just doing non sequiturs now? I didn’t defend image generation tools in the slightest.

> We need to be able to express our idea, feelings, our perception of the world in ways that do not fit corporate standards of text, audio, or visual communication.

I agree. My point started and ended with “van Gogh in an awful example when talking about artist who ‘made it’”. That’s it. There is nothing in there to be extrapolated to AI or any other subject.

So don't use the tool or make your own. You don't get to tell people what to use or like.

> Then stop selling it as a tool to replace humans

I don't understand why people assume that the purpose of any tool is to "replace humans". Automation doesn't replace humans and never has and never will. It simply does certain tasks that humans used to do, freeing people up to do different tasks. There is not a limited amount of work that can be done, there isn't a limited amount of _creative_ work that can be done. Even if AIs were good enough to do every creative task done by humans today (and they aren't and won't be any time soon), that doesn't mean that humans will have nothing of value to produce, or that humans will have been "replaced". There is always going to be work for humans to do, even in a universe where AI have super human capabilities at all tasks.

In particular, human beings strongly value the opinions and creative output of _human beings_ simply for the reason that they are human and similar to them. That will never change, no matter how intelligent that AIs get.

> I don't understand why people assume that the purpose of any tool is to "replace humans".

Because of this https://news.ycombinator.com/item?id=40070566