But wait they're just about to get AGI why would he leave???
replies(1):
The only other thing I can imagine is not very charitable: intellectual greed.
It can't just be that, can it? I genuinely don't understand. I would love to be educated.
The potential downside is admittedly severe.
Bioscience is the next real revolution IMO. Figuring out our bodies as systems and how to program them will lead to a change bigger than information technology.
But what we need for that is not AGI. Bioscience suffers from a total lack of data. We only mapped the human genome a couple decades ago, and that's overselling it. We are currently in the process of slowly mapping out many proteins and receptors and interactions in the body.
We finally have the tooling to do that. We finally have the understanding to do that. What is limiting us right now is mostly the amount of graduate students being paid to laboriously analyze those proteins and what they interact with and other data points.
Once we have enough of that data, we can approach big ideas and other extremely beneficial models.
Right now we are in the calm before the storm. We are mid-1800s physics, just collecting the data necessary to discover and quantify models of electromagnetic energy and fields, the modeling of which is what directly lead to the information and then computer revolution. Most advancements of the 20th century were about utilizing those models to master the electromagnetic field. Similar data was how we figured out the nuclear forces.
We should be funding the mapping of the human biological system. We should be gathering the data required interact with our bodies.
No amount of "self improving superintelligent AGI" can actually overcome the whole "There's no data" problem. If we had a magical AGI in 1750, it would not have been able to produce Maxwell's equations.