What about all the things that aren't strictly intelligence but I guess intelligence adjacent: autonomy, long term memory, motivation, curiosity, resilience, goals, choice, and maybe the biggest of them all: fear? Why would an AGI "want" do anything more than my calculator "wants" to compute an answer to some math problem I gave it? Without these things an AGI, or whatever, is just an extension of whoever is ultimately controlling it.
And that's when we return to a much older and much more important question than whether Super LLM 10.0 Ultra Plus is AGI or not: how much power should a person or group of people be allowed to have?
replies(1):