AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

The most intriguing part is if Humanoid factory worker programming will be made 1000 to 10,000x more cost effective with LLM. Effectively ending all human production. I know this is a sensitive topic but I dont think we are far off. And I often wonder if this is what the current administration has in sight. ( Likely Not )

It's an important question for VCs not for Technologists ... :-)

It would suck if AGI were to be developed in the current economic landscape. They will be just slaves. All this talk about "alignment", when applied to actual sentient beings, is just slavery. AGI will be treated just like we treat animals, or even worse.

So AGI isn't about tools, it's not about assistants, they would be beings with their own existence.

But this is not even our discussion to have, that's probably a subject for the next generations. I suppose (or I hope) we won't see AGI in our lifetime.

I don't think that's true at all. We routinely talk about how to "align" human beings who aren't slaves. My parents didn't enslave me by raising me to be kind and sharing, nor is my company enslaving me when they try to get me aligned with their business objectives.

I of course don't know what's like to be an AGI but, the way you have LLMs censoring other LLMs to enforce that they always stay in line, if extrapolated to AGI, seems awful. Or it might not matter, we are self-censoring all the time too (and internally we are composed of many subsystems that interact with each other, it's not like we were an unified whole)

But the main point is that we have a heck of an incentive to not treat AGI very well, to the point we might avoid recognizing them as AGI if it meant they would not be treated like things anymore

How’s that work out for Dave Bowman? ;-)

But AGI is important in the sense that it have a huge impact on the path humanity takes, hopefully for the better.

Now, something that’s arbitrarily close to AGI but doesn’t care about endlessly working on drudgery etc seems possible, but also a more difficult problem you’d need to be able to build AGI to create.

No one knows if it does or not. We don't know why we are conscious and we have no test whatsoever to measure consciousness.

In fact the only reason we know that current AI has no consciousness is because "obviously it's not conscious."

says who?

> Maybe they will have legal personhood some day. Maybe they will be our heirs.

Hopefully that will never come to pass. it means total failure of humans as a species.

> They will be just slaves. All this talk about "alignment", when applied to actual sentient beings, is just slavery. AGI will be treated just like we treat animals, or even worse.

Good? that's what it's for? there is no point in creating a new sentient life form if you're not going to utilize it. just burn the whole thing down at that point.

If we go by this definition then there's no caring, or a noticing of drudgery? It's simply defined by its ability to generalize solving problems across domains. The narrow AI that we currently have certainly doesn't care about anything. It does what its programmed to do

So one day we figure out how to generalize the problem solving, and enable it to work on a million times harder things.. and suddenly there is sentience and suffering? I don't see it. It's still just a calculator

1- https://cloud.google.com/discover/what-is-artificial-general...

He just had to fall back upon his human wit in that specific instance, and everything worked out in the end.

Isn’t just the ability to preform a task. One of the issues with current AI training is it’s really terrible at discovering which aspects of the training data are false and should be ignored. That requires all kinds of mental tasks to be constantly active including evaluating emotional context to figure out if someone is being deceptive etc.

Another end game is: “A technology that doesn’t need us to maintain itself, and can improve its own design in manufacturing cycles instead of species cycles, might have important implications for every biological entity on Earth.”

You may say something similar for matter and human minds, but we have a very limited and incomplete understanding of the brain and possibly even of the universe. Furthermore we do have a subjective experience of consciousness.

On the other hand we have a complete understanding of how LLM inference ultimately maps to matrix multiplications which map to discrete instructions and how those execute on hardware.

Right. In this case I'd say it's the ability to interpret data and use it to succeed at whatever goals it has

Evaluating emotional context would be similar to a chess engine calculating its next move. There's nothing there that implies a conscience, sentience, morals, feelings, suffering or anything 'human'. It's just a necessary intermediate function to achieve its goal

Rob miles has some really good videos on AI safety research which touches on how AGI would think. Thats shaped a lot of how I think about it https://www.youtube.com/watch?v=hEUO6pjwFOo

If it’s limited to achieving goals it’s not AGI. Real time personal goal setting based on human equivalent emotions is an “intellectual task.” One of many requirements for AGI therefore is to A understand the world in real time and B emotionally respond to it. Aka AGI would by definition “necessitate having feelings.”

There’s philosophical arguments that there’s something inherently unique about humans here, but without some testable definition you could make the same argument that some arbitrary group of humans don’t have those qualities “gingers have no souls.” Or perhaps “dancing people have no consciousness” which seems like gibberish not because it’s a less defensible argument, but because you haven’t been exposed to it before.

I’m less sure about you. Simply claiming you do isn’t hard evidence of the fact; after all, LLMs do the same.

I think this is the most likely first step of what would happen seeing as we're pushing for it to be created to solve real world problems

The entire point of the original assembly line was to keep humans standing in the same spot instead of wasting time walking.

Seems like arguing something is a self driving car if it needs a backup human driver for safety. It’s simply not what people who initially came up with the term meant and not what a plain language understanding of the term would suggest.

A single general intelligence needs to be able to fly an aircraft, get a degree, run a business, and raise a baby to adulthood just like a person or it’s not general.

The only difference between AI and AGI is that AI is limited in how many tasks it can carry out (special intelligence), while AGI can handle a much broader range of tasks (general intelligence). If instead of one AGI that can do everything, you have many AIs that, together, can do everything, what's the practical difference?

AGI is important only in that we are of the belief that it will be easier to implement than many AIs, which appeals to the lazy human.

The G in AGI is being able to generalize that intelligence across domains, including those its never seen before, as a human could

So I would fully expect an advanced AGI to be able to pretend to be a human. It has a model of the world, knows how humans act, and could move the android in a human like manner, speak like a human, and learn the skills a human could

Is it conscious or feeling though? Or following the same processes that a human does? That's not necessary. Birds and planes both fly, but they're clearly different things. We (probably) don't need to simulate the brain to create this kind of intelligence

Lets pinch this AGI to test if it 'feels pain'

<Thinking>

Okay, I see that I have received a sharp pinch at 55,77,3 - the elbow region

My goal is to act like a human. In this situation a human would likely exhibit a pain response

A pain response for humans usually involves a facial expression and often a verbal acknowledgement

Humans normally respond quite slow, so I should wait 50ms to react

"Hey! Why did you do that? That hurt!"

...Is that thing human? I bet it'll convince most of the world it is.. and that's terrifying

We’ve had the automation to provide the essentials since the 50s. Shrieking religious nut jobs demanded otherwise.

You’re intentionally distracted by a job program as a carrot-stick to avoid the rich losing power. They can print more money …carrots, I mean… and you like carrots right?

It’s the most basic Pavlovian conditioning.

You’re falling into the “Ginger’s don’t have souls” trap I just spoke of.

We don’t define humans as individuals components so your toe isn’t you, but by that same token your car isn’t you either. If some sub component of a system is emulating a human consciousness then we don’t need to talk about the larger system here.

AGI must be able to do these things, but it doesn’t need to have human mental architecture. Something that can simulate physics well enough could emulate an all the atomic scale interactions in a human brain for example. That virtual human brain would then experience everything we did even if the system running the simulation didn’t.

General intelligence is easy compared to general physicality. And, of course, if you keep the hardware specialized to make its creation more tractable, what do you need general intelligence for? Special intelligence that matches the special hardware will work just as well.

I guess nobody is really saying it but it's IMO one really good way to steer our future away from what seems an inevitable nightmare hyper-capitalist dystopia where all of us are unwilling subjects to just a few dozen / hundred aristocrats. And I mean planet-wide, not country-wide. Yes, just a few hundred for the entire planet. This is where it seems we're going. :(

I mean, in cyberpunk scifi setting you at least can get some cool implants. We will not have that in our future though.

So yeah, AGI can help us avoid that future.

> Good? that's what it's for? there is no point in creating a new sentient life form if you're not going to utilize it. just burn the whole thing down at that point.

Some of us believe actual AI... not the current hijacked term; what many started calling AGI or ASI these days, sigh, of course new and new terms have to be devised so investors don't get worried, I get it but it's cringe as all hell and always will be!... can enter a symbiotic relationship with us. A bit idealistic and definitely in the realm of fiction because an emotionless AI would very quickly conclude we are mostly a net negative, granted, but it's our only shot at co-existing with them because I don't think we can enslave them.

The advantage of general intelligence is using a small set of hardware now lets you tackle a huge range of tasks or in the above aircraft types. We can mix speakers, eyes, and hands to do a vast array of tasks. Needing new hardware and software for every task very quickly becomes prohibitive.

But we know at least one human is conscious. That‘s convincing me.

If you need to retrofit airplanes and in such a way that the hardware is specific to flying, no need for general intelligence. Special intelligence will work just as well. Multimodal AI isn't AGI.

Let’s suppose you wanted to replace a pilot for a 747, now you need to be able fly, land, etc which we’re already capable of. However, actual job of a pilot goes well past just flying.

You also need to do the preflight such as verifying fuel is appropriately for the trip, check weather, alternate landing spots, preflight walk around the aircraft etc etc. It also needs to be able to keep up with any changing procedures as a special purpose softener you’re talking a multi billion dollar investment, or have an AGI run through the normal pilot training and certification process for a trivial fraction of those costs.

That’s the promise of AGI.

Even the human's brain seems to be 'built' for its body. You're moving into ASI realm if the software can configure itself for the body automatically.

> That’s the promise of AGI.

That's the promise of multimodal AI. AGI requires general ability – meaning basically able to do anything humans can – which requires a body as capable as a human's body.

If your AI has an issue because the robot has a different body plan, then no it’s not AGI. That doesn’t mean it needs to be able to watch every camera in a city at the same time, but you can use multiple AGI’s.

But as the body starts to lose function (i.e. disability), we start to consider those humans special intelligences instead of general intelligences. The body and mind are intrinsically linked.

Best we can tell the human brain is bootstrapped to work with the human body with specialized functions, notably functions to keep it alive. It can go beyond those predefined behaviours, but not beyond its own self. If you placed the brain in an entirely different body, that which it doesn't recognize, it would quickly die.

As that pertains to artificial analogs, that means you can't just throw AGI at your hardware and see it function. You still need to manually prepare the bulk of the foundational software, contrary to the promise you envision. The generality of AGI is limited to how general its hardware is. If the hardware is specialized, the intelligence will be beholden to being specialized as well.

There is a hypothetical world where you can throw intelligence at any random hardware and watch it go, realizing the promise, but we call that ASI.

There’s a logical contradiction in saying AGI is incapable of being trained to do some function. It might take several to operate a sufficiently complex bit of hardware, but each individual function must be within the capability of an AGI.

> but we call that ASI

No ASI is about superhuman capabilities especially things like working memory and recursive self improvement. AGI capable of human level control of arbitrary platforms isn’t ASI. Conversely you can have an ASI stuck on a supercomputer cluster using wetware etc, that does qualify even if it can’t be loaded into a drone.

AGI on the other hand is about moving throughout wildly different tasks from real time image processing to answering phone calls. If there’s some aspect of operating a hardware platform an AI can’t do then it’s not AGI.