Measuring the impact of AI on experienced open-source developer productivity

(metr.org)

690 points dheerajvs | 1 comments | 10 Jul 25 16:29 UTC | HN request time: 0.206s | source

Show context

atleastoptimal ◴[10 Jul 25 18:52 UTC] No.44524249[source]▶

I’m not surprised that AI doesn’t help people with 5+ years experience in open source contribution, but I’d imagine most people aren’t claiming AI tools are at senior engineer level yet.

Soon once the tools and how people use them improve AI won’t be a hinderance for advanced tasks like this, and soon after AI will be able to do these prs on their own. It’s inevitable given the rate of improvement even since this study.

replies(1): >>44524341 #

artee_49 ◴[10 Jul 25 19:02 UTC] No.44524341[source]▶

>>44524249 #

Even for senior levels the claim has been that AI will speed up their coding (take it over) so they can focus on higher level decisions and abstract level concepts. These contributions are not those and based on prior predictions the productivity should have gone up.

replies(1): >>44527017 #

1. atleastoptimal ◴[10 Jul 25 23:57 UTC] No.44527017[source]▶

>>44524341 #

It would be different I'm sure if they were making contributions to repos they had less familiarity with. In my experience and talking with those who use AI most effectively, it is best leveraged as a way of getting up to speed or creating code for a framework/project you have less familiarity with. The general ratio determining the effectiveness of non-AI coding vs AI coding is the familiarity the user has with the codebase * the complexity of the codebase : the amount of closed-loop abstractions in the tasks the coder needs to carry out.

Currently AI is like a junior engineer, and if you don't have good experience managing junior engineers, AI isn't going to help you as much.

↑