Jagged AGI: o3, Gemini 2.5, and everything after

It sounds like you have a different definition of "general" in the context of intelligence from the one I shared. What is it?

General intelligence means it can do the same intellectual tasks as humans can, including learning to do different kinds of intellectual jobs. Current AI can't learn to do most jobs like a human kid can, so its not AGI.

This is the original definition of AGI. Some data scientists try to move the goalposts to something else and call something that can't replace humans "AGI".

This is a very simple definition that is easy to see when it is fulfilled because then companies can operate without humans.

What intellectual tasks can humans do that language models can't? Particularly agentic language model frameworks.

A normal software engineering job? You have access to email and can send code etc. No current model manages anything close to that. Even much simpler jobs can't be automated like that by them.

So basically any form of longer term tasks cannot be done by them currently. Short term tasks with constant supervision is about the only things they can do, and that is very limited, most tasks are long term tasks.

> You have access to email and can send code etc. No current model manages anything close to that.

This is an issue of tooling, not intelligence. Language models absolutely have the power to process email and send (push?) code, should you give them the tooling to do so (also true of human intelligence).

> So basically any form of longer term tasks cannot be done by them currently. Short term tasks with constant supervision is about the only things they can do, and that is very limited, most tasks are long term tasks.

Are humans that have limited memory due to a condition not capable of general intelligence, xor does intelligence exist on a spectrum? Also, long term tasks can be decomposed into short term tasks. Perhaps automatically, by a language model.

Have you actually tried agentic LLM based frameworks that use tool calling for long term memory storage and retrieval, or have you decided that because these tools do not behave perfectly in a fluid environment where humans do not behave perfectly either, that it's "impossible"?

Why would it be a tooling issue? AI has access to email, IDEs, and all kinds of systems. It still cannot go and build software on its own by speaking to stakeholders, taking instructions from a PM, understanding it needs to speak to DevOps to release its code, suggesting to product team that feature is better developed as part of core product, objecting to SA about the architecture, and on and on…

(If it was a tooling issue, AGI could build the missing tools)

> Have you actually tried agentic LLM based frameworks that use tool calling for long term memory storage and retrieval, or have you decided that because these tools do not behave perfectly in a fluid environment where humans do not behave perfectly either, that it's "impossible"?

i.e. "Have you tried this vague, unnamed thing that I alude to that seems to be the answer that contradicts your point, but actually doesn't?"

AGI = 90% of software devs, psychotherapists, lawyers, teachers lose their jobs, we are not there.

Once LLMs can fork themselves, reflect and accumulate domain specific knowledge and transfer the whole context back to the model weights, once that knowledge can become more important than the pre-pretrained information, once they can form new neurons related to a project topic, then yes, we will have AGI (probably not that far away). Once LLM's can keep trying to find a bug for days and weeks and months, go through the debugger, ask people relevant questions, deploy code with new debugging traces, deploy mitigations and so on, we will have AGI.

Otherwise, AI is stuck in this groundhog day type scenario, where it's forever the brightest intern that any company has ever seen, but he's forever stuck at day 0 on the job, forever not that usefull, but full of potential.

Weird spiky things that are hard to characterise even within one specific model, and where the ability to reliably identify such things itself causes subsequent models to not fail so much.

A few months ago, I'd have said "create image with coherent text"*, but that's now changed. At least in English — trying to get ChatGPT's new image mode to draw the 狐 symbol sometimes works, sometimes goes weird in the way latin characters used to.

* if the ability to generate images doesn't count as "language model" then one intellectual task they can't do is "draw images", see Simon Willison's pelican challenge: https://simonwillison.net/tags/pelican-riding-a-bicycle/

Read a bunch of books not present in the training data on a specific topic, and learn something from it.

You can cheat with tooling like RAG or agentic frameworks, but the result isn't going to be good and it's not the AI that learns.

But besides this fundamental limitation, had you tried implementing production ready stuff with LLM, you'd have discovered that language models are still painfully unreliable even for the tasks they are supposed to be good at: they will still hallucinate when summarizing, fail to adhere to the prompt, add paragraphs in English at random when working in French, edit unrelated parts of the code you ask it to edit, etc, etc.

You can work around many of those once you've identified it, but that still counts as a fail in a response to your question.

> Have you actually tried agentic LLM based frameworks that use tool calling for long term memory storage and retrieval,

You can work around the limitations of LLMs' intelligence with your own and an external workflow you design, but I don't see how that counts as part of the LLM's intelligence.

> This is an issue of tooling, not intelligence. Language models absolutely have the power to process email and send (push?) code, should you give them the tooling to do so (also true of human intelligence).

At a certain point, a tooling issue becomes an intelligence issue. AGI would be able to build the tools they need to succeed.

If we have millions of these things deployed, they can work 24/7, and they supposedly have human-level intelligence, then why haven't they been able to bootstrap their own tooling yet?

Humans have general intelligence. A network of humans have better general intelligence.

LLMs have general intelligence. A network of LLMs have better general intelligence.

If a single language model isn't intelligent enough for a task, but a human is, there is a good chance there exists a sufficient network of language models that is intelligent enough.

> LLMs have general intelligence.

No they don't. That's the key part you keep assuming without justification. Interestingly enough you haven't responded to my other comment [1].

You asked “What intellectual tasks can humans do that language models can't?” and now that I'm thinking about it again, I think the more apt question would be the reverse:

“What intellectual tasks can a LLM do autonomously without any human supervision (direct or indirect[2]) if there's money at stake?”

You'll see that the list is going to be very shallow if not empty.

> A network of LLMs have better general intelligence.

Your argument was about tool calling for long term memory, this isn't “a network of LLM” but an LLM another tool chosen by a human to deal with LLM's limitations one one particular problem (and of you need long term memory for another problem you're very likely to need to rework both your prompt and your choice of tools to address it: it's not the LLM that solves it but your own intelligence).

[1]: https://news.ycombinator.com/item?id=43755623 [2] indirect supervision would be the human designing an automatic verification system to check LLMs output before using it. Any kind of verification that is planned in advance by the human and not improvised by the LLM when facing the problem counts as indirect supervision, even if it relies on another LLM.