←back to thread

AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

(www.dwarkesh.com)
174 points Philpax | 5 comments | 17 Apr 25 16:42 UTC | HN request time: 0s | source
Show context
ksec ◴[17 Apr 25 17:44 UTC] No.43720025[source]▶
Is AGI even important? I believe the next 10 to 15 years will be Assisted Intelligence. There are things that current LLM are so poor I dont believe a 100x increase in pref / watt is going to make much difference. But it is going to be good enough there wont be an AI Winter. Since current AI has already reached escape velocity and actually increase productivity in many areas.

The most intriguing part is if Humanoid factory worker programming will be made 1000 to 10,000x more cost effective with LLM. Effectively ending all human production. I know this is a sensitive topic but I dont think we are far off. And I often wonder if this is what the current administration has in sight. ( Likely Not )

replies(9): >>43720094 #>>43721244 #>>43721573 #>>43721593 #>>43721933 #>>43722074 #>>43722240 #>>43723605 #>>43726461 #
nextaccountic ◴[17 Apr 25 20:07 UTC] No.43721593[source]▶
AGI is important for the future of humanity. Maybe they will have legal personhood some day. Maybe they will be our heirs.

It would suck if AGI were to be developed in the current economic landscape. They will be just slaves. All this talk about "alignment", when applied to actual sentient beings, is just slavery. AGI will be treated just like we treat animals, or even worse.

So AGI isn't about tools, it's not about assistants, they would be beings with their own existence.

But this is not even our discussion to have, that's probably a subject for the next generations. I suppose (or I hope) we won't see AGI in our lifetime.

replies(5): >>43721770 #>>43722215 #>>43722462 #>>43722548 #>>43723075 #
AstroBen ◴[17 Apr 25 21:06 UTC] No.43722215[source]▶
Why does AGI necessitate having feelings or consciousness, or the ability to suffer? It seems a bit far to be giving future ultra-advanced calculators legal personhood?
replies(2): >>43722245 #>>43722350 #
Retric ◴[17 Apr 25 21:10 UTC] No.43722245[source]▶
The general part of general intelligence. If they don’t think in those terms there’s an inherent limitation.

Now, something that’s arbitrarily close to AGI but doesn’t care about endlessly working on drudgery etc seems possible, but also a more difficult problem you’d need to be able to build AGI to create.

replies(1): >>43722577 #
AstroBen ◴[17 Apr 25 21:45 UTC] No.43722577[source]▶
Artificial general intelligence (AGI) refers to the hypothetical intelligence of a machine that possesses the ability to understand or learn any intellectual task that a human being can. Generalization ability and Common Sense Knowledge [1]

If we go by this definition then there's no caring, or a noticing of drudgery? It's simply defined by its ability to generalize solving problems across domains. The narrow AI that we currently have certainly doesn't care about anything. It does what its programmed to do

So one day we figure out how to generalize the problem solving, and enable it to work on a million times harder things.. and suddenly there is sentience and suffering? I don't see it. It's still just a calculator

1- https://cloud.google.com/discover/what-is-artificial-general...

replies(3): >>43722727 #>>43722760 #>>43726476 #
Retric ◴[17 Apr 25 22:04 UTC] No.43722727[source]▶
“ability to understand”

Isn’t just the ability to preform a task. One of the issues with current AI training is it’s really terrible at discovering which aspects of the training data are false and should be ignored. That requires all kinds of mental tasks to be constantly active including evaluating emotional context to figure out if someone is being deceptive etc.

replies(1): >>43723272 #
AstroBen ◴[17 Apr 25 23:20 UTC] No.43723272[source]▶
> Isn’t just the ability to preform a task.

Right. In this case I'd say it's the ability to interpret data and use it to succeed at whatever goals it has

Evaluating emotional context would be similar to a chess engine calculating its next move. There's nothing there that implies a conscience, sentience, morals, feelings, suffering or anything 'human'. It's just a necessary intermediate function to achieve its goal

Rob miles has some really good videos on AI safety research which touches on how AGI would think. Thats shaped a lot of how I think about it https://www.youtube.com/watch?v=hEUO6pjwFOo

replies(1): >>43724697 #
1. Retric ◴[18 Apr 25 03:23 UTC] No.43724697[source]▶
> Evaluating emotional context would be similar to a chess engine calculating its next move. There's nothing there that implies a conscience, sentience, morals, feelings, suffering or anything 'human'. It's just a necessary intermediate function to achieve its goal

If it’s limited to achieving goals it’s not AGI. Real time personal goal setting based on human equivalent emotions is an “intellectual task.” One of many requirements for AGI therefore is to A understand the world in real time and B emotionally respond to it. Aka AGI would by definition “necessitate having feelings.”

There’s philosophical arguments that there’s something inherently unique about humans here, but without some testable definition you could make the same argument that some arbitrary group of humans don’t have those qualities “gingers have no souls.” Or perhaps “dancing people have no consciousness” which seems like gibberish not because it’s a less defensible argument, but because you haven’t been exposed to it before.

replies(1): >>43725398 #
2. AstroBen ◴[18 Apr 25 05:55 UTC] No.43725398[source]▶
I mean we just fundamentally have different definitions of AGI. Mine's based on outcomes and what it can do, so purely goal based. Not the processes that mimic humans or animals

I think this is the most likely first step of what would happen seeing as we're pushing for it to be created to solve real world problems

replies(1): >>43727574 #
3. Retric ◴[18 Apr 25 12:49 UTC] No.43727574[source]▶
I’m not sure how you can argue something is a general intelligence if it can’t do those kinds of things? Comes out of the factory with a command: “Operate this android for a lifetime pretending to be human.”

Seems like arguing something is a self driving car if it needs a backup human driver for safety. It’s simply not what people who initially came up with the term meant and not what a plain language understanding of the term would suggest.

replies(1): >>43728468 #
4. AstroBen ◴[18 Apr 25 14:35 UTC] No.43728468{3}[source]▶
Because I see intelligence as the ability to produce effective actions towards a goal. A more intelligent chess AI beats a less intelligent one by making better moves towards the goal of winning the game

The G in AGI is being able to generalize that intelligence across domains, including those its never seen before, as a human could

So I would fully expect an advanced AGI to be able to pretend to be a human. It has a model of the world, knows how humans act, and could move the android in a human like manner, speak like a human, and learn the skills a human could

Is it conscious or feeling though? Or following the same processes that a human does? That's not necessary. Birds and planes both fly, but they're clearly different things. We (probably) don't need to simulate the brain to create this kind of intelligence

Lets pinch this AGI to test if it 'feels pain'

<Thinking>

Okay, I see that I have received a sharp pinch at 55,77,3 - the elbow region

My goal is to act like a human. In this situation a human would likely exhibit a pain response

A pain response for humans usually involves a facial expression and often a verbal acknowledgement

Humans normally respond quite slow, so I should wait 50ms to react

"Hey! Why did you do that? That hurt!"

...Is that thing human? I bet it'll convince most of the world it is.. and that's terrifying

replies(1): >>43728907 #
5. Retric ◴[18 Apr 25 15:21 UTC] No.43728907{4}[source]▶
> Is it conscious or feeling though?

You’re falling into the “Ginger’s don’t have souls” trap I just spoke of.

We don’t define humans as individuals components so your toe isn’t you, but by that same token your car isn’t you either. If some sub component of a system is emulating a human consciousness then we don’t need to talk about the larger system here.

AGI must be able to do these things, but it doesn’t need to have human mental architecture. Something that can simulate physics well enough could emulate an all the atomic scale interactions in a human brain for example. That virtual human brain would then experience everything we did even if the system running the simulation didn’t.