AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu

It is science fiction to think that a system like a computer can behave at all like a brain. Computers are incredibly rigid systems with only the limited variance we permit. "Software" is flexible in comparison to creating dedicated circuits for our computations but is nothing by comparison to our minds.

Ask yourself, why is it so hard to get a cryptographically secure random number? Because computers are pure unadulterated determinism -- put the same random seed value in your code and get the same "random numbers" every time in the same order. Computers need to be like this to be good tools.

Assuming that AGI is possible in the kinds of computers we know how to build means that we think a mind can be reduced to a probabilistic or deterministic system. And from my brief experience on this planet I don't believe that premise. Your experience may differ and it might be fun to talk about.

In Aristotle's ethics he talks a lot about ergon (purpose) -- hammers are different than people, computers are different than people, they have an obvious purpose (because they are tools made with an end in mind). Minds strive -- we have desires, wants and needs -- even if it is simply to survive or better yet thrive (eudaimonia).

An attempt to create a mind is another thing entirely and not something we know how to start. Rolling dice hasn't gotten anywhere. So I'd wager AGI somewhere in the realm of 30 years to never.

It's clear that your argument is based on feels and you're using philosophy to make it sound more legitimate.

A lot of things that humans believed were true due to their brief experience on this planet ended up being false: earth is the center of the universe, heavier objects fall faster than lighter ones, time ticked the same everywhere, species are fixed and unchanging.

So what your brief experience on this planet makes you believe has no bearing on what is correct. It might very well be that our mind can be reduced to a probabilistic and deterministic system. It might also be that our mind is a non-deterministic system that can be modeled in a computer.

Also does this also mean that you believe that brain emulations (uploads) are not possible, even given an arbitrary amount of compute power?

That said, I don't think computers need to be human to have an emergent intelligence. It can be different in kind if not in degree.

Then you've missed the part of software.

Software isn't computer science, it's not always about code. It's about solving problems in a way we can control and manufacture.

If we needed random numbers, we could easily use a hardware that uses some physics property, or we could pull in an observation from an api like the weather. We don't do these things because pseudo-random is good enough, and other solutions have drawbacks (like requiring an internet for api calls). But that doesn't mean software can't solve these problems.

For generic stuff you probably can't tell the difference, but once you move to the edges you start to hear the steps in digital vs the smooth transition of analog.

In the same way, AI runs on bits and bytes, and there's only so much detail you can fit into that.

You can approximate reality, but it'll never quite be reality.

I'd be much more concerned with growing organic brains in a lab. I wouldn't be surprised to learn that people are covertly working on that.

Take the same model weights give it the same inputs, get the same outputs. Same with the pseudo-random number generator. And the "same inputs" is especially limited versus what humans are used to.

What's the machine code of an AGI gonna look like? It makes one illegal instruction and crashes? If if changes tboughts will it flush the TLB and CPU pipeline? ;) I jest but really think about the metal. The inside of modern computers is tightly controlled with no room for anything unpredictable. I really don't think a von Neumann (or Harvard ;) machine is going to cut it. Honestly I don't know what will, controlled but not controlled, artificially designed but not deterministic.

In fact, that we've made a computer as unreliable as a human at reproducing data (ala hallucinating/making s** up) is an achievement itself, as much of an anti-goal as it may be. If you want accuracy, you don't use a probabilistic system on such a wide problem space (identify a bad solder joint from an image, sure. Write my thesis, not so much)

If so, what do you think about the concept of a human "hear[ing] the steps" in a digital playback system using a sampling rate of 192kHz, a rate at which many high-resolution files are available for purchase?

How about the same question but at a sampling rate of 44.1kHz, or the way a normal "red book" music CD is encoded?

It's such a small leap to see how an artificial intelligence can/could become capable of understanding and have intentionality.

That is one hell of a network, and it can all operate fully in parallel while continuously training itself. Computers have gotten pretty good at doing things in parallel, but not that good.

;-)

This makes computers incredibly good at what people are not good at -- predictably doing math correctly, following a procedure, etc.

But because all of the possibilities of the computer had to be written up as circuitry or software beforehand, it's variability of outputs is constrained to what we put into it in the first place (whether that's a seed for randomness or model weights).

You can get random numbers and feed it into the computer but we call that "fuzzing" which is a search for crashes indicating unhandled input cases and possible bugs or security issues.

Right now the guess is that it will be mostly a bunch of multiplications and additions.

> It makes one illegal instruction and crashes?

And our hearth quivers just slightly the wrong way and we die. Or a tiny blood cloth plugs a vessel in our brain and we die. Do you feel that our fragility is a good reason why meat cannot be intelligent?

> I jest but really think about the metal.

Ok. I'm thinking about the metal. What should this thinking illuminate?

> The inside of modern computers is tightly controlled with no room for anything unpredictable.

Let's assume we can't make AGI because we need randomness and unpredictability in our computers. We can very easily add unpredictability. The simple and stupid solution is to add some sensor (like a camera CCD) and stare at the measurement noise. You don't even need a lens on that CCD. You can cap it so it sees "all black", and then what it measures is basically heat noise of the sensors. Voila. Your computer has now unpredictability. People who actually make semiconductors probably can come up with even simpler and easier ways to integrate unpredictability right on the same chip we compute with.

You still haven't really argued why you think "unpredictableness" is the missing component of course. Beside the fact that it just feels right to you.

https://en.wikipedia.org/wiki/Hardware_random_number_generat...

(and if we assume that non-determinism is randomness, non-deterministic brain could be simulated by software plus an entropy source)

My comment around digital vs analog is more of an analogy around producing sounds rather than playing back samples though.

There's a Masterclass with Joel Zimmerman (DeadMau5) where he explains the stepping effect when it comes to his music production. Perhaps he just needs a software upgrade, but there was a lesson where he showed the stepping effect which was audibly noticeable when comparing digital vs analog equipment.

It is science fiction to think that a plane could act at all like a bird. Although... it doesn't need to in order to fly

Intelligence doesn't mean we need to recreate the brain in a computer system. Sentience, maybe. General intelligence no

There are an infinite number of frequencies between two points - point 'a' and point 'b'. What I'm talking about are the "steps" you hear as you move across the frequency range.

Let's say that the shortest interval at which our hearing has good frequency acuity (say, as good as it can be) is 1 second.

In this interval, we have 44100 samples.

Let's imagine the samples graphically: a "44K" pixel wide image.

We have some waveform across this image. What is the smallest frequency stretch or shrink that will change the image? Note: not necessarily be audible, but just change the pixels.

If we grab one endpoint of the waveform and move it by less than half a pixel, there is no difference, right? We have to stretch it by a whole pixel.

Let's assume that some people (perhaps most) can hear that difference. It might not be true, but it's the weakest assumption.

That's a 0.0023 percent difference!

One cent (1/100th of a semitone) is a 0.058% difference: so the difference we are considering is 25 X smaller.

I really don't think you can hear 1/25 of a cent difference in pitch, over interval of one second, or even longer.

Over shorter time scales less than a second, the resolution in our perception of pitch gets worse.

E.g. when a violinist is playing a really fast run, you don't notice it if the notes have intonation that is off. The longer "landing" notes in the solo have to be good.

When bad pitch is slight, we need not only longer notes, but to hear it together with other notes, because the beats between them are an important clue (and in fact the artifact we will find most objectionable).

Pre digital technology will not have frequency resolution which is that good. I don't think you can get tape to move at a speed that stays within 0.0023 percent of a set target. In consumer tape equipment, you can hear audible "wow" and "flutter" as the tape speed oscillates. When the frequency of a periodic signal wobbles, you get new signals in there: side bands.

I don't think that there is any perceptible aspect of sound that is not captured in the ordinary consumer sample rates and sample resolutions. I suspect 48 kHz and 24 bits is way past diminishing returns.

I'm curious what it is that Deadmau5 thinks he discovered, and under what test conditions.

If you feed that true randomness into a computer, what use is it? Will it impair the computer at the very tasks in which it excels?

> That all inputs should be defined and mapped to some output and that this process is predictable and reproducible.

Suppoise we sample a precise 10,000.00 kHz analog signal (sinusoid) and speed up the sampled signal by 0.0023 percent. It will have a frequency of 10,000.23 Hz.

The f2 - f2 difference between them is 0.23 Hz, which means if they are mixed together, we will hear beats at 0.46 Hz: a little slower than once in every two seconds.

So in this contrived way, where we have the original source and the digitized one side by side, we can obtain an audible effect correlating to the steps in resolution of the sampling method.

I'm guessing Deadmau5 might have set up an experiment along these lines.

Musicians tend to be oblivious to something like 5 cent errors in the intonations of their instruments, in the lower registers. E.g. world renowned guitarists play on axes that have no nut compensation, without which you can't even get close to accurate intontation.

I mean, humans aren't exactly good at generating random numbers either.

And of course, every Intel and AMD CPU these days has a hardware random number generator in it.

This comment thread is dull. I'm bailing out.

The distance to that beginning in time is approx 13 billion years. There is no approximation of distance to the beginning because the space is created at that point and continues to be created.

Imagine the Earth being on the surface of a sphere and so asking what is the center of the surface of a sphere? The sphere has a center but on the surface there is no center.

At least this is my understanding of how to approach these kind of questions.

We understand the building blocks of the brain pretty well. We know the structure and composition of neurons, we know how they are connected, what chemicals flow through them and how all these chemicals interact, and how that interaction translates to signal propagation. In fact, the neural networks we use in computing are loosely modelled on biological neurons. Both models are essentially comprised of interconnected units where each unit has weights to convert its incoming signals to outgoing signals. The predominant difference is in how these units adjust their weights, where computational models use back propagation and gradient descent, biological models use timing information from voltage changes.

But just because we understand the science of something perfectly well, doesn't mean we can precisely predict how something will work. Biological networks are very, very complex systems comprising of billions of neurons with trillions of connections acting on input that can vary in immeasurable number of ways. It's like predicting earthquakes. Even though we understand the science behind plate tectonics, to precisely predict an earthquake we need to map the properties of every inch of continental plates which is an impossible task. But doesn't mean we can't use the same scientific building blocks to build simulations of earthquakes which behave like any real earthquake would behave. If it looks like a duck, quacks like a duck, then what is a duck?

Mechanically it's different since humans are not such advanced mechanics as nature, but of course comparing the whole brain function to a simple flight is a bit silly

2. Computers can't do continuous and unsupervised learning, which means computers require structured input, labeled data, and predefined objectives to learn anything. Humans learn passively all the time just by existing in the environment

I think it's less about the randomness and more about that all the functionality of a computer is defined up front, in software, in training, in hardware. Sure you can add randomness and pick between two paths randomly but a computer couldn't spontaneously pick to go down a path that wasn't defined for it.

1. Self-rewiring is just a matter of hardware design. Neuromorphic hardware is a thing.

2. LLM foundation models are actually unsupervised in a way, since they simply take any arbitrary text and try to complete it. It's the instruction fine-tuning that is supervised. (Q/A pairs)

The point is about unsupervised learning. Once an LLM is trained, its weights are frozen — it won’t update itself during a chat. Prompt-driven Inference is immediate, not persistent, you can define a term or concept mid-chat and it will behave as if it learned it, but only until the context window ends. If it was the other way all models would drift very quickly.

It very much can. Jump scares, deep grief are known to cause heart attacks. It is called stress cardiomyopathy. Or your meatsuit can indiredtly do that by ingesting the wrong chemicals.

> If you could make an intelligent process, what would it think of an operating system kernel

Idk. What do you think of your hypothalamus? It can make you unconscious at any time. It in fact makes you unconscious about once a day. Do you fear it? What if one day it won’t wake you up? Or what if it jacks up your internal body temperature and cooks you alive from the inside? It can do that!

Now you might say you don’t worry about that, because through your long life your hypothalamus proved to be reliable. It predictably does what it needs to do, to keep you alive. And you would be right. Your higher cognitive functions have a good working relationship with your lower level processes.

Similarly for an AGI to be inteligent it needs to have a good working relationship with the hardware it is running on. That means that if the kernel is temperamental and idk descheduling the higher level AGI process then the AGI will mallfunction and not appear that inteligent. Same as if you meet Albert Einstein while he is chemically put to sleep. He won’t appear inteligent at all! At best he will be just drooling there.

> Can you imagine an intelligent process in such a place, as static representation of data in ram?

Yes. You can’t? This is not really a convincing argument.

> It all sounds frankly ridiculous.

I think what you are doing is that you are looking at implementation details and feeling a disconnect between that and the possibility of inteligence. Do you feel the same ridiculousnes about a meatblob doing things and appearing inteligent?

> a computer couldn't spontaneously pick to go down a path that wasn't defined for it.

Can you?

> Yes. You can’t? This is not > really a convincing argument.

Fair, I believe it's called begging the question. But for some context is that people of many recent technological ages have talked about the brain like a piece of technology -- e.g. like a printing press, a radio, a TV.

I think we've found what we've wanted to find (a hardware-software dichotomy in the brain) and then occasionally get surprised when things aren't all that clearly separated. So with that in mind, I personally without any particularly good evidence to the contrary am not of the belief that your brain can be represented as a static state. Pribram's holonomic mind theory comes to mind as a possible way brain state could have trouble being represented in RAM.( https://en.m.wikipedia.org/wiki/Holonomic_brain_theory)

> ...you are looking at implementation details and feeling a disconnect between that and the possibility of inteligence. Do you feel the same ridiculousnes about a meatblob doing things and appearing inteligent?

If I was a biologist I might. My grandfather was a microbiologist and scoffed at my atheism. But with a computer at least the details are understandable and knowable being created by people. We haven't cracked the consciousness of a fruit fly despite having a map of it's brain.

>> a computer couldn't spontaneously pick to go down a path that wasn't defined for it.

> Can you?

Love it. I re-read Fight Club recently, it's a reasonable question. The worries of determinism versus free will still loom large in this sort of world view. We get a kind of "god in the gaps" type problem with free will being reduced down to the spaces where you don't have an explanation.