Human

Perhaps I am unimaginative about whatever AGI might be, but it so often feels to me like predictions are more based on sci-fi than observation. The theorized AI is some anthropomorphization of a 1960s mainframe: you tell it what to do and it executes that exactly with precise logic and no understanding of nuance or ambiguity. Maybe it is evil. The SOTA in AI at the moment is very good at nuance and ambiguity but sometimes does things that are nonsensical. I think there should be less planning around something super-logical.

We already see that at smaller scales though with other machine learning algorithms; image recognition algorithms will lock on to any consistency in your training set more often than learning to recognize what you actually want it to [0]. It's not a huge stretch to map that pattern out to a more generally intelligent system having a poorly defined reward function doing really weird stuff.

[0] Like the tumor recognition algorithm that instead learned to recognize rulers or the triage algorithm that decided asthma patients had BETTER outcomes with pulmonary diseases not making the connection that it's because they get higher priority care - https://venturebeat.com/business/when-ai-flags-the-ruler-not...