(metr.org)

688 points dheerajvs | 2 comments | 10 Jul 25 16:29 UTC | HN request time: 0.756s | source

Show context

narush ◴[10 Jul 25 17:28 UTC] No.44523346[source]▶

Hey HN, study author here. I'm a long-time HN user -- and I'll be in the comments today to answer questions/comments when possible!

If you're short on time, I'd recommend just reading the linked blogpost or the announcement thread here [1], rather than the full paper.

[1] https://x.com/METR_Evals/status/1943360399220388093

replies(7): >>44523757 #>>44523844 #>>44523891 #>>44524187 #>>44524724 #>>44524983 #>>44528188 #

1. JackC ◴[10 Jul 25 20:10 UTC] No.44524983[source]▶

>>44523346 #

(I read the post but not paper.)

Did you measure subjective fatigue as one way to explain the misperception that AI was faster? As a developer-turned-manager I like AI because it's easier when my brain is tired.

replies(1): >>44525029 #

2. narush ◴[10 Jul 25 20:14 UTC] No.44525029[source]▶

>>44524983 (TP) #

We attempted to! We explore this more in the section Trading speed for ease (C.2.5) in the paper (https://metr.org/Early_2025_AI_Experienced_OS_Devs_Study.pdf).

TLDR: mixed evidence that developers make it less effortful, from quantitative and qualitative reports. Unclear effect.

↑

Measuring the impact of AI on experienced open-source developer productivity