Problems the AI industry is not addressing adequately

> "This is purely an observation: You only jump ship in the middle of a conquest if either all ships are arriving at the same time (unlikely) or neither is arriving at all. This means that no AI lab is close to AGI."

The central claim here is illogical.

The way I see it, if you believe that AGI is imminent, and if your personal efforts are not entirely crucial to bringing AGI about (just about all engineers are in this category), and if you believe that AGI will obviate most forms of computer-related work, your best move is to do whatever is most profitable in the near-term.

If you make $500k/year, and Meta is offering you $10M/year, then you ought to take the new job. Hoard money, true believer. Then, when AGI hits, you'll be in a better personal position.

Essentially, the author's core assumption is that working for a lower salary at a company that may develop AGI is preferable to working for a much higher salary at a company that may develop AGI. I don't see how that makes any sense.

Being part of the team that achieved AGI first would be to write your name in history forever. That could mean more to people than money.

Also 10m would be a drop in the bucket compared to being a shareholder of a company that has achieved AGI; you could also imagine the influence and fame that comes with it.

*some people

>your best move is to do whatever is most profitable in the near-term

Unless you’re a significant shareholder, that’s almost always the best move, anyway. Companies have no loyalty to you and you need to watch out for yourself and why you’re living.

Kind of a sucker move here since you personally will 100% be forgotten. We are only going to remember one or two people who did any of this. Say Sam Altman and Ilya Sttsveker. Everyone else will be forgotten. The authors or the Transformer paper are unlikely to make it into the history books or even popular imagination. Think about the Manhattan Project. We recently made a movie remembering that one guy who did something on the Manhattan Project, but he will soon fade back into obscurity. Sometimes people say that it was about Einstein's theory of relativity. The only people who know who folks like Ulam were are physicists. The legions of technicians who made it all come together are totally forgotten. Same with the space program or the first computer or pretty much any engineering marvel.

Well depends on what you value. Achieving/contributing to something impactful first is for many people valuable even if it doesn't come with fame. Historically, this mindframe has been popular especially amongst scientists.

Personally I think the ones who will be remembered will be the ones who publish useful methods first, not the ones who succeed commercially.

It'll be Vaswani and the others for the transformer, then maybe Zelikman and those on that paper for thought tokens, then maybe some of the RNN people and word embedding people will be cited as pioneers. Sutskever will definitely be remembered for GPT-1 though, being first to really scale up transformers. But it'll actually be like with flight and a whole mass of people will be remembered, just as we now remember everyone from the Wrights to Bleriot and to Busemann, Prandtl, even Whitcomb.

> Being part of the team that achieved AGI first would be to write your name in history forever. That could mean more to people than money.

Uh, sure. How many rocket engineers who worked for moon landing could you name?

Is "we" the particular set of scientists who know those last four people? Surely you realize they're nowhere near as famous as the Wright brothers, right? This is giving strong https://xkcd.com/2501/ feelings.

Yes, that is indeed the 'we', but I think more people are knowledgeable than is obvious.

I'm not an aerodynamicist, and I know about those guys, so they can't be infinitely obscure. I imagine every French person knows about Bleriot at least.

I read that most of the crazy comp Zuck is offering is in stock. So in a way, going to the place where they have lots of stock reflects their belief about where AGI is going to happen first.

"The grass is greener elsewhere" isn't inconsistent with a belief that AGI will happen somewhere.

It means you don't have much faith that the company you're working at will be the ones to pull it off.

Comp is comp, no matter how it comes (though the details can vary in important ways).

I know people who've taking quite good comp from startups to do things that would require fundamental laws of physics to be invalidated; they took the money and devised experiments that would show the law to be wrong.

How many new species of infinite chattel slave did they invent?

With a salary of $10m/year, handwave roughly half of that goes to taxes, you'd be making just shy of $100k post-tax per week. Call me a sellout, but goddamn. For that much money, there's a lot of places I could be convinced to put my faith into that I wouldn't otherwise.

Facebook is already public, so they can sell the day it vests and get it in cold hard cash in their bank account. If Facebook weren't public it would be a more interesting point as they couldn't liquidate immediately, but they can, so I wouldn't read anything into that.

It might buy loyalty for a while, but after it accumulates, for many people it would be "why am I even working at all" money.

And if they don't like their boss and the other job sounds better, well...

But maybe the salary is also higher?

I'm an avgeek with a MSc in engineering. I vaguely recall the name Bleriot from physics, although I have no clue what he actually did. I have never even heard the names Busemann, Prandtl, or Whitcomb.

I find this super surprising, because even I who don't do aerodynamics I still know about thes guys.

Bleriot was a french aviation pioneer and not a physicist. He built the first monoplane. Busemann was an aerodynamicist who invented wing sweep and also did important work on supersonic flight. Prandtl is known for research on lift distribution over wings, wingtip vortices, induced drag and he basically invented much of the theory about wings. Whitcomb gave his name to the Whitcomb area rule, although Otto Frenzl had come up with it earlier during WWII.

What is wing sweep, what is induced drag, what is the area rule?

Airliners don't have the wings going straight out, instead being swept back. You can also sweep them forward to get the same effect, but you will rarely want to do that due to other problems. This means that the cross sectional area of the aircraft varies less along the length and reduces wave drag.

If there's no lift there's no pressure different between the upper side of the wing and the lower side of the wing. But if there's lift there's higher pressure on the bottom and lower on top, so air wants to flow around the wing, from bottom to top, producing a wingtip vortex. This flow creates drag, and this drag is called lift-induced drag or just 'induced drag'.

The area rule is about minimizing wave drag by keeping the cross sectional area of different parts of the aircraft close to the cross sectional area of the corresponding cross-section of a minimal drag body. It leads to wing sweep and certain fuselage shapes.