Problems the AI industry is not addressing adequately

I love how much the proponents is this tech are starting to sound like the opponents.

What I can't figure out is why this author thinks it's good if these companies do invent a real AGI...

""" I’m basically calling the AI industry dishonest, but I want to qualify by saying they are unnecessarily dishonest. Because they don’t need to be! They should just not make abstract claims about how much the world will change due to AI in no time, and they will be fine. They undermine the real effort they put into their work—which is genuine!

Charitably, they may not even be dishonest at all, but carelessly unintrospective. Maybe they think they’re being truthful when they make claims that AGI is near, but then they fail to examine dispassionately the inconsistency of their actions.

When your identity is tied to the future, you don’t state beliefs but wishes. And we, the rest of the world, intuitively know. """

He's not saying either way, just pointing out that they could just be honest, but that might hamper their ability to beg for more money.

But that isn't my point. Regardless of whether they're honest, have we even agreed that "AGI" is good?

Everyone is so tumbling over themselves even to discuss will-it-won't-it, but they seem to think about it like some kind of Manhattan project or Space race.

Like, they're *so sure* it's gonna take everyone's jobs so that there will be nothing left for people other than a life of leisure. To me this just sounds like the collapse of society, but apparently the only thing worse would be if China got the tech first. Oh no, they might use it to collapse their society!

Somebody's math doesn't add up.

"Carelessly unintrospective" becomes dishonest when you allow other people to rely on your words. Carelessly unintrospective is a tolerable interpersonal position, it is a nearly fraudulent business position.