←back to thread

Measuring the impact of AI on experienced open-source developer productivity

(metr.org)
688 points dheerajvs | 1 comments | 10 Jul 25 16:29 UTC | HN request time: 0.207s | source
Show context
Jabrov ◴[10 Jul 25 16:45 UTC] No.44522918[source]▶
Very interesting methodology, but the sample size (16) is way too low. Would love to see this repeated with more participants.
replies(2): >>44523093 #>>44523724 #
narush ◴[10 Jul 25 17:59 UTC] No.44523724[source]▶
Noting that most of our power comes from the number of tasks that developers complete; it's 246 total completed issues in the course of this study -- developers do about 15 issues (7.5 with AI and 7.5 without AI) on average.
replies(2): >>44524047 #>>44524680 #
1. biophysboy ◴[10 Jul 25 19:33 UTC] No.44524680[source]▶
Did you compare the variance within individuals (due to treatment) to the variance between individuals (due to other stuff)?