A Research Preview of Codex

This is a bit of a game theory problem. "Training senior engineers" is an expensive and thankless task: you bear essentially all the cost, and most of the total benefit accrues to others as a positive externality. Griping at companies that they should undertake to provide this positive externality isn't really a constructive solution.

I think some people are betting on the fact that AI can replace junior devs in 2-5 years and seniors in 10-20, when the old ones are largely gone. But that's sort of beside the point as far as most corporate decision-making.

With Agentic RL training and sufficient data, AI operating at the level of average senior engineers should become plausible in a couple to a few years.

Top-tier engineers who integrate a deep understanding of business and user needs into technical design will likely be safe until we get full-fledged AGI.

That sounds like a dangerous bet.

Sounds like a bet a later CEO will need to check.

As I see it, it's actually the only safe bet.

Case 1: you keep training engineers.

Case 1.1: AGI soon, you don't need juniors or seniors besides a very few. You cost yourself a ton of money that competitors can reinvest into R&D, use to undercut your prices, or return to keep their investors happy.

Case 1.2: No AGI. Wages rise, a lot. You must remain in line with that to avoid losing those engineers you trained.

Case 2: You quit training juniors and let AI do the work.

Case 2.1: AGI soon, you have saved yourself a bundle of cash and remain mostly in in line with the market.

Case 2.2: no AGI, you are in the same bidding war for talent as everyone else, the same place you'd have been were you to have spent all that cash to train engineers. You now have a juicier balance sheet with which to enter this bidding war.

The only way out of this, you can probably see, is some sort of external co-ordination, as is the case with most of these situations. The high-EV move is to quit training juniors, by a mile, independently of whether AI can replace senior devs in a decade.

This hyper-fixation on replacing engineers in writing code is hilarious, and dangerous, to me. Many people, even in tech companies, have no idea how software is built, maintained, and run.

I think instead we should focus on getting rid of managers and product owners.

The real judge will be survivorship bias and as a betting man, I might think product owners are the ones with the entrepreneurial spirit to make it to the other side.

it’s obviously intensely correlated: the vast majority of scenarios either both are replaced or neither

I think it'll be great if you're working in software not for a software company.

As a dev, if you try taking away my product owners I will fight you. Who am I going to ask for requirements and sign-offs, the CEO?

Your architect, principal engineer etc. (one spot-on job title I've seen is "product architect"), who in turn talks to the senior management. Basically an engineer with a talent and experience for building products rather than a manager with superficial understanding of engineering. I think the most ambitious teams have someone like this on top - or at least around

Perhaps the role will merge into one, and will replace a good chunk of those jobs.

E.g.:

If we have 10 PMs and 90 devs today, that could be hypothetically be replace by 8 PM+Dev, 20 specialized devs, and 2 specialized PMs in the future.

An interesting thing to consider is that Codex might get people to be better at delegating, which might improve the effectiveness of hiring junior engineers. Because the senior engineers will have better skills at delegating, leading to a more effective collaboration.

You’re looking at it from the point of view of an individual company. I’m seeing it as a risk for the entire industry.

Senior engineers are already very well paid. Wages rising a lot from where they already are, while companies compete for a few people, and those who can’t afford it need to lean on AI or wait 10+ years for someone to develop with equivalent expertise… all of this sounds bad for the industry. It’s only good for the few senior engineers that are about to retire, and the few who went out of their way to not use AI and acquire actual skills.

Why in a few years? What training data is missing that we can’t have senior level agents today?

I've worked for a company which turned from startup to this. Product owners had no clue what they own. And no brain capacity to suggest something useful. They were just taken from the street at best, most likely had relatives' helping hands. In a couple of years company probably tripled manages headcount. It didn't help.

A 70% reduction in the labor force of product and engineering has a lot of consequences.

The people who will come out the other side are domain focused people with the engineering chops to understand the system end to end, and the customer skills to understand what needs to be built.

Well, yes. But nobody is running the entire industry. You’re running a company that has competitors willing to eat your lunch.

Product owners and project managers have the soft skills to convince the company that they aren't a drain on its resources regardless of what they actually are.

> Case 1.2: No AGI. Wages rise, a lot. You must remain in line with that to avoid losing those engineers you trained.

No you don't. Most engineers are shy, conflict-averse, and hate change. You can keep underpaying them and most of them will stay.

If you have 10PMs and 90 devs today, and go to 8 "hybrid" PMs + 2 specialized PMs, you're probably still creating backlog items faster than that team can close them.

So you end up with some choices:

* do you move at the same speed, with fewer people?

* do you try to move faster, with less of a reduction in people? this could be trickier than it sounds because if the frequency of changes increases the frequency of unintended consequences likely does too, so your team will have to spend time reacting to that

I think the companies that win will be the second batch. It's what happens today, basically, but today you have to convince VCs or the public market to give you a bunch of more money to hire to 10x the team size. Getting a (one-off?) chance to do that through tooling improvements is a big gift, wasting it on reducing costs instead of increasing growth could be risky.

Case 1.3: No AGI, tools increase productivity a lot, you have a bigger team and you make them more productive. In the meantime, while everyone else was scared of hiring, you got a bunch of stuff done to gain a lead in the market.

You get high EV because everyone else in your market voluntarily slowing down is a gift-wrapped miracle for you.

(Even in an AGI-soon case - you spent a bit more (let's be serious here, we're not talking about spending our entire bankroll on 18months of new hires here) in short term to get ahead, then you shift people around or lay them off. Your competitors invested that money into R&D? What does that even mean if it didn't involve hiring and AGI happens soon anyway?)

----

(Case 3: AGI soon, you don't need yourself anymore - it's hard to imagine a sufficiently advanced "AGI" that someone only replaces software devs but leaves the structure, management, and MBA-trappings of modern exchange and businesses alone.)

Yes. everyone will eventually have the job title of "problem solver"