Can someone more informed than me comment — is it me, or does it seem that Situational Awareness essay rings more and more true?
replies(1):
https://situational-awareness.ai/parting-thoughts/
There are some big differences with the atom bomb though :
A lot of people have been talking about AGI / the Singularity for decades, and that essay is yet another example.
Not sure if it's the hindsight talking, but the situation with (un)controlling fission was much clearer ? We knew it was at least theoretically possible. And the first test was very precisely engineered (in fact switching at the last minute to a more complex design because the calculations showed that the material wasn't pure enough to work). And the very first test worked.
Meanwhile, despite for AGI it (almost) all being out in the open, nobody can define what AGI exactly is, and it's all extremely empirical : throw compute at the wall, try another algorithm, see if the black box seems smarter, or not. Nobody knows even in theory when (or even if) AGI will be reached, and for how long scaling up compute will keep providing the very impressive results we have seen so far.