←back to thread

Measuring the impact of AI on experienced open-source developer productivity

(metr.org)
688 points dheerajvs | 1 comments | 10 Jul 25 16:29 UTC | HN request time: 0s | source
Show context
tcdent ◴[10 Jul 25 17:15 UTC] No.44523208[source]▶
This study neglects to incorporate the fact that I have forgotten how to write code.
replies(2): >>44523645 #>>44524138 #
1. narush ◴[10 Jul 25 18:39 UTC] No.44524138[source]▶
Honestly, this is a fair point -- and speaks the difficulty of figuring out the right baseline to measure against here!

If we studied folks with _no_ AI experience, then we might underestimate speedup, as these folks are learning tools (see a discussion of learning effects in section (C.2.7) - Below-average use of AI tools - in the paper). If we studied folks with _only_ AI experience, then we might overestimate speedup, as perhaps these folks can't really program without AI at all.

In some sense, these are just two separate and interesting questions - I'm excited for future work to really dig in on both!