A Research Preview of Codex

It looks we are in this interesting cycle: millions of engineers contribute to open-source on github. The best of our minds use the code to develop powerful models to replace exactly these engineers. In fact, the more code a group contributes to github, the easier it is for the companies to replace this group. Case in point, frontend engineers are impacted most so far.

Does this mean people will be less incentivized to contribute to open source as time goes by?

P.S., I think the current trend is a wakeup call to us software engineers. We thought we were doing highly creative work, but in reality we spend a lot of time doing the basic job of knowledge workers: retrieving knowledge and interpolating some basic and highly predictable variations. Unfortunately, the current AI is really good at replacing this type of work.

My optimistic view is that in long term we will have invent or expand into more interesting work, but I'm not sure how long we will have to wait. The current generation of software engineers may suffer high supply but low demand of our profession for years to come.

> P.S., I think the current trend is a wakeup call to us software engineers. We thought we were doing highly creative work, but in reality we spend a lot of time doing the basic job of knowledge workers: retrieving knowledge and interpolating some basic and highly predictable variations. Unfortunately, the current AI is really good at replacing this type of work.

Most of the waking hours of most creative work have this type of drudgery. Professional painters and designers spend most of their time replicating ideas that are well fleshed-out. Musicians spend most of their time rehearsing existing compositions.

There is a point to be made that these repetitive tasks are a prerequisite to come up with creative ideas.

> doing the basic job of knowledge workers

If you extrapolate and generalize further... what is at risk is any task that involves taking information input (text, audio, images, video, etc.), and applying it to create some information output or perform some action which is useful.

That's basically the definition of work. It's not just knowledge work, it's literally any work.

As much as I support community developed software and "free as in freedom", "Open Source" got completely perverted into tricking people to work for free for huge financial benefits for others. Your comment is just one example of that.

For that reason all my silly little side projects are now in private repos. I dont care the chance somebody builds a business around them is slim to none. Dont think putting a license will protect you either. You'd have to know somebody is violating your license before you can even think about doing anything and that's basically impossible if it gets ripped into a private codebase and isnt obvious externally.

I disagree. AI have shown to most capable in what we consider creative jobs. Music creation, voice acting, text/story writing, art creation, video creation and more.

> "Open Source" got completely perverted into tricking people to work for free for huge financial benefits for others

I'm quite conflicted on this assessment. On one hand, I was wondering if we would get better job market if there were not much open-sourced systems. We may have had a much slower growth, but we would see our growth last for a lot more years, which mean we may enjoy our profession until our retirement and more. On the other hand, open source did create large cakes, right? Like the "big data" market, the ML market, the distributed system market, and etc. Like the millions of data scientists who could barely use Pandas and scipy, or hundreds of thousands of ML engineers who couldn't even bother to know what semi positive definite matrix is.

Interesting times.

If you mean create as in literally, sure. But not in being creative. AI can't solve novel problems yet. The person you're replying to obviously means being creative not literally creating something.

What is the qualifier for this? Didn't one of the models recently create a "novel" algorithm for a math problem? I'm not sure this holds water anymore.

> Does this mean people will be less incentivized to contribute to open source as time goes by?

personally, I completely stopped 2 years ago

it's the same as the stack overflow problem: the incentive to contribute tends towards zero, at which point the plagiarism machine stops improving

> AI have shown to most capable in what we consider creative jobs

no it creates shit thats close enough for people who are in a rush and dont care.

ie, you need artwork for shit on temu, boom job done.

You want to make a poster for a bake sale, boom job done.

Need some free music that sounds close enough to be swifty, but not enough to get sued, great.

But as an expression of creativity, most people cant get it to do that.

Its currently slightly more configurable clipart.

Now do open science.

More generally, specialty knowledge is valuable. From now on, all employees will be monitored in order to replace them.

You can't say AI is creating something new but that it isn't being creative with clearly explaining why you think that's the case. AI is creating novel solution to problems humans haven't cracked in centuries. I don't see anything more creative than this.

> AI creates novel algorithms beating thousands of googlers.

Random HNer on an AI post one day later

> Its currently slightly more configurable clipart.

It's so ridiculous at this point that I can just laugh about this.

Comparing music and algorithms is nonsense.

AI music is disposable generic soulless trash, even if it is technically correct, in accordance with the rules and conventions of music theory. AI generates Muzak. Totally generic and derivative.

There is no AI equivalent to Kurt Cobain, or James Brown, or Tori Amos.

> Does this mean people will be less incentivized to contribute to open source as time goes by?

Yes. I certainly don't intend to put any free code online until I can legally bar AI bros from using it without payment. As Mike Monteiro put it long ago, "F** you, pay me" (https://www.youtube.com/watch?v=jVkLVRt6c1U)

> From now on, all employees will be monitored in order to replace them.

This is going on a t-shirt.

Two people from different backgrounds have differing opinions on different systems.

This just in, the response was also to an entirely different context.

> rowanG077: Holy shit look how different the opinions are! LOL HN is so wild.

"After a couple of million suggestions and a few dozen repetitions of the overall process"

How much of it is brute force?

> AI is creating novel solution to problems humans haven't cracked in centuries

Really?

Protect you from what?

What harm is there to you if someone uses some of your code to build a business, as compared to not doing so? How are you worse off?

I’ve never understood this mentality. It seems very zero sum and kind of anti social. I’ve built a couple of businesses, and there’s always economic or technical precedent. I honestly don’t mind paying it forward if someone can benefit from side projects I enjoyed doing anyways.

Serious deja vu to the late 70’s and early 80’s when “real musicians” said exactly the same thing about synthesizers and drum machines.

The difference, of course, being that synthesizers and drum machines are instruments that require actual skill and talent and can be used to express the unique musical style of an artist, whereas AI requires neither skill nor talent, and it cannot generate anything with actual artistic direction, intent or innovation, much less a unique creative style.

AI is never going to give the world a modern Kraftwerk or Silver Apples or Brian Eno. The best an AI "artist" can do is have the machine mimic them.

Exactly. If you are not actively going to compete in that space why not let someone else compete instead using your work?

Are we maybe moving the goalposts here? How much of human creativity is brute force?

So let's say your side project improves your life by 5 happiness points. You have two options:

--- OPTION A - Keep your project private.

• You get five happiness points.

--- OPTION B - Make your project public.

• Other individuals may get a small number of happiness points.

• A megacorp might turn your project into a major product without compensating you and get a million happiness points.

• You get five happiness points.

----------

In either scenario, you still end up with five happiness points. If you release your code, other people may get even more happiness points than you, which isn't really fair. But you are no worse off, and you've increased humanity's total wealth of happiness points.

You really dont see why somebody wouldnt like a megacorp to take their hard work, use it to make a billion dollars, dont see a cent themselves, while struggling to buy a house in this very unaffordable housing market?

https://knowyourmeme.com/memes/total-happiness-in-the-world-...

This assumes that none of the effects of making a project public or private have any impact on the output of your personal utility function, which may be true for you personally, but certainly cannot validly be assumed to be generally true.

Those are instruments played and sequenced by humans. Meaningless comparison.

Look, in the hands of a skilled artist, generative fill is really useful.

In the same way that the synth is superstitious is banging.

Sampling, when in the hands of a legend is also spectacular, see the prodigy and a break down of the samples they used. (or any half decent hiphop band)

Then you get akon who just sped up a single sample put a beat on it and shat out some halfarsed shit.

> What harm is there to you if someone uses some of your code to build a business, as compared to not doing so? How are you worse off?

This someone might be someone I dislike. It would cause me some mild annoyance that they benefited from my effort.

Google, Microsoft, Meta, IBM, Red Hat, etc. are huge players in open source, they probably contribute significantly more hours of work in building and maintaining major open source projects than the hobbyists.

Not that hobbyists don't contribute, but these models are certainly being trained on the work of salaried engineers as much as their trained on hobbyists' spare time projects.

Marble sculptures, that's what I came up with.

There is no AI equivalent to Kurt Cobain or any other artists because 99% of what they are is not at all about their musical skill but all about marketing. There are thousands of musicians just as skilled if not more than Kurt Cobain or James Brown yet who don't have their fame. I also have no doubt an AI will outperform most musicians in short order in the foreseeable future. The step from making no music at all, to making acceptable music is gigantic compared to making acceptable music to making great music.

Yes, really. Just yesterday google announced their AI was able to improve on human SotA algorithms in 25% of the cases fed into it. One of them was 4x4 complex matrix multiply. Which had pretty huge pressure to be improved.

Still the same thing. The argument then was that synths weren’t “real instruments” and that sequencers meant people weren’t “real musicians”.

AI relies on prompting. In the hands of a skilled artist it is just another tool. In the hands of an amateur hack, it is no different than giving a drum kit to a 4 year old.

The tool is not the art. It never was.

It’s not healthy to let hypothetical spite guide your participation in civilization.

It's also not healthy to sip on whiskey or smoke some tobacco, but I do both on occasion.

Spite is also pretty pleasurable.

You have a right to the work you author. If you are selling your work or your services, then you have to protect your product - yourself.

If you decide to put open source out there for others to use, then sure. You chose to allow that.

If you didn't license it for that, then it shouldn't be in an LLM.

Yea those companies do...but you know what? Those engineers working there contributing that code are getting paid to do so.

They were right in many cases. You can choose to pick out the small percentage of musicians who were successful there or you can recognize the many that were never known.

You can do the same for photography.

People keep lowering expectations or demands on quality because things get easier and humans always prefer the easy option.

Same. It'll be interesting to see when open source slows down enough and is maybe all but eliminated...how will things advance?

Once everyone uses AI. How will anything new come about?

AI could quite possibly slow down innovation and improvement.

If that someone then takes that work that you're providing for free to other people to build on it, makes a closed source product out of it and gives you no attribution, then you can be darn well sure I want to protect it.

People might also lace open source with Trojans and back doors. It's going to get interesting.

And much as people joke about "exposure" not putting food on the table, being able to walk into a job interview with your name known because the company is already using your code/project is huge.

My pessimistic view is that we're liable to end up cutting off the pipeline into the industry. Similar to lawyers replacing clerks with bots, if senior engineers can now command bots rather than mentor new hires, where is the on-ramp? How does one actually gain enough experience to become a senior?

Or is all this a nothing-burger, since the new hires will just be commanding bots of their own, but on lower level tasks that they are qualified to supervise?

I'm skeptical. I wouldn't call brute-force creating, not really. And Google obviously has an interest in promoting AI.

Not to say it's not impressive and good.

No, we're trying to have a conversation.