←back to thread

Measuring the impact of AI on experienced open-source developer productivity

(metr.org)
688 points dheerajvs | 1 comments | 10 Jul 25 16:29 UTC | HN request time: 0s | source
Show context
NewsaHackO ◴[10 Jul 25 17:15 UTC] No.44523209[source]▶
So they paid developers 300 x 246 = about 73K just for developer recruitment for the study, which is not in any academic journal, or has no peer reviews? The underlying paper looks quite polished and not overtly AI generated so I don't want to say it entirely made up, but how were they even able to get funding for this?
replies(5): >>44523366 #>>44523443 #>>44523493 #>>44523508 #>>44523611 #
narush ◴[10 Jul 25 17:36 UTC] No.44523443[source]▶
Our largest funding was through The Audacious Project -- you can see an announcement here: https://metr.org/blog/2024-10-09-new-support-through-the-aud...

Per our website, “To date, April 2025, we have not accepted compensation from AI companies for the evaluations we have conducted.” You can check out the footnote on this page: https://metr.org/donate

replies(1): >>44523551 #
iLoveOncall ◴[10 Jul 25 17:45 UTC] No.44523551[source]▶
This is really disingenuous when you also say that OpenAI and Anthropic have provided you with access and compute credits (on https://metr.org/about).

Not all payment is cash. Compute credits is still by all means compensation.

replies(4): >>44523633 #>>44524298 #>>44524647 #>>44530608 #
golly_ned ◴[10 Jul 25 19:30 UTC] No.44524647{3}[source]▶
Those are compute credits that are directly spent on the experiment itself. It's no more "compensation" than a chemistry researcher being "compensated" with test tubes.
replies(1): >>44524898 #
1. iLoveOncall ◴[10 Jul 25 20:02 UTC] No.44524898{4}[source]▶
> Those are compute credits that are directly spent on the experiment itself.

You're extrapolating, it's not saying this anywhere.

> It's no more "compensation" than a chemistry researcher being "compensated" with test tubes.

Yes, that's compensation too. Thanks for contributing another example. Here's another one: it's no more compensation than a software engineer being compensated with a new computer.

Actually the situation here is way worse than your example. Unless the chemistry researcher is commissioned by Big Test Tube Corp. to conduct research on the outcome of using their test tubes, there's no conflict of interest here. But there is an obvious conflict of interest on AI research being financed by credits given by AI companies to use their own AI tools.