Problems the AI industry is not addressing adequately

> "This is purely an observation: You only jump ship in the middle of a conquest if either all ships are arriving at the same time (unlikely) or neither is arriving at all. This means that no AI lab is close to AGI."

The central claim here is illogical.

The way I see it, if you believe that AGI is imminent, and if your personal efforts are not entirely crucial to bringing AGI about (just about all engineers are in this category), and if you believe that AGI will obviate most forms of computer-related work, your best move is to do whatever is most profitable in the near-term.

If you make $500k/year, and Meta is offering you $10M/year, then you ought to take the new job. Hoard money, true believer. Then, when AGI hits, you'll be in a better personal position.

Essentially, the author's core assumption is that working for a lower salary at a company that may develop AGI is preferable to working for a much higher salary at a company that may develop AGI. I don't see how that makes any sense.

Being part of the team that achieved AGI first would be to write your name in history forever. That could mean more to people than money.

Also 10m would be a drop in the bucket compared to being a shareholder of a company that has achieved AGI; you could also imagine the influence and fame that comes with it.

Kind of a sucker move here since you personally will 100% be forgotten. We are only going to remember one or two people who did any of this. Say Sam Altman and Ilya Sttsveker. Everyone else will be forgotten. The authors or the Transformer paper are unlikely to make it into the history books or even popular imagination. Think about the Manhattan Project. We recently made a movie remembering that one guy who did something on the Manhattan Project, but he will soon fade back into obscurity. Sometimes people say that it was about Einstein's theory of relativity. The only people who know who folks like Ulam were are physicists. The legions of technicians who made it all come together are totally forgotten. Same with the space program or the first computer or pretty much any engineering marvel.

Personally I think the ones who will be remembered will be the ones who publish useful methods first, not the ones who succeed commercially.

It'll be Vaswani and the others for the transformer, then maybe Zelikman and those on that paper for thought tokens, then maybe some of the RNN people and word embedding people will be cited as pioneers. Sutskever will definitely be remembered for GPT-1 though, being first to really scale up transformers. But it'll actually be like with flight and a whole mass of people will be remembered, just as we now remember everyone from the Wrights to Bleriot and to Busemann, Prandtl, even Whitcomb.

Is "we" the particular set of scientists who know those last four people? Surely you realize they're nowhere near as famous as the Wright brothers, right? This is giving strong https://xkcd.com/2501/ feelings.

Yes, that is indeed the 'we', but I think more people are knowledgeable than is obvious.

I'm not an aerodynamicist, and I know about those guys, so they can't be infinitely obscure. I imagine every French person knows about Bleriot at least.

I'm an avgeek with a MSc in engineering. I vaguely recall the name Bleriot from physics, although I have no clue what he actually did. I have never even heard the names Busemann, Prandtl, or Whitcomb.