←back to thread

Survey: a third of senior developers say over half their code is AI-generated

(www.fastly.com)
214 points Brajeshwar | 4 comments | 31 Aug 25 14:55 UTC | HN request time: 0s | source
Show context
Rochus ◴[01 Sep 25 09:18 UTC] No.45090991[source]▶
The article claims, that senior developers with over 10 years of experience are more than twice as likely to heavily rely on AI tools compared to their junior counterparts. No p-values or statistical significance tests are reported in either The Register article or Fastly's original blog post.

I have over 30 years of experience and recently used Claude Opus 4.1 (via browser and claude.ai) to generate an ECMA-335 and an LLVM code generator for a compiler, and a Qt adapter for the Mono soft debugging protocol. Each task resulted in 2-3kLOC of C++.

The Claude experience was mixed; there is a high probability that the system doesn't respond or just quickly shows an overloaded message and does nothing. If it generates code, I quckly run in some output limitation and have to manually press "continue", and then often the result gets scrambled (i.e. the order of the generated code fragments gets mixed up, which requires another round with Claude to fix).

After this process, the resulting code then compiled immediately, which impressed me. But it is full of omissions and logical errors. I am still testing and correcting. All in all, I can't say at this point that Claude has really taken any work off my hands. In order to understand the code and assess the correctness of the intermediate results, I need to know exactly how to implement the problem myself. And you have to test everything in detail and do a lot of redesigning and correcting. Some implementations are just stubs, and even after several attempts, there was still no implementation.

In my opinion, what is currently available (via my $20 subscription) is impressive, but it neither replaces experience nor does it really save time.

So yes, now I'm one of the 30% seniors who used AI tools, but I didn't really benefit from them in these specific tasks. Not surprisingly, also the original blog states, that nearly 30% of senior developers report "editing AI output enough to offset most of the time savings". So not really a success so far. But all in all I'm still impressed.

replies(5): >>45091343 #>>45091757 #>>45092344 #>>45092985 #>>45099223 #
epolanski ◴[01 Sep 25 12:53 UTC] No.45092344[source]▶
Imho your post summarizes 90% of the posts I see about AI coding on HN: not understanding the tools, not understanding their strenghts and weaknesses, not being good at prompting or context management yet forming strong(ish) opinions.

If you don't know what they are good at and how to use them of course you may end up with mixed results and yes, you may waste time.

That's a criticism I have also towards AI super enthusiasts (especially vibe coders, albeit you won't find much here), they often confuse the fact that LLMs often one shot 80% of the solutions with the idea that LLMs are 80% there, whereas the Pareto principle well applies to software development where it's the hardest 20% that's gonna prove difficult.

replies(4): >>45092402 #>>45092707 #>>45099984 #>>45137386 #
1. Rochus ◴[01 Sep 25 13:04 UTC] No.45092402[source]▶
I'm pretty good at prompting and I successfully use Perplexity (mostly with Claude Sonnet 4) to develop concepts, sometimes with the same session expanded over several days. I think the user interface is much superior over Claude.ai. My hope was that the newer Claude Opus 4.1 would be much better in solving complicated coding tasks, which doesn't seem to be the case. For this I had to subscribe to claude.ai. Actually I didn't see much difference in performance, but a much worse UI and availability experience. When it comes to developing a complex topic in a factual dialogue, Claude Sonnet Thinking seems to me to be even more suitable than Claude Opus.
replies(1): >>45092771 #
2. epolanski ◴[01 Sep 25 14:04 UTC] No.45092771[source]▶
I'll be more detailed in my second reply.

1) Your original post asks a lot if not too much out the LLM, the expectation you have is too big, to the point that to get anywhere near decent results you need a super detailed prompt (if not several spec documents) and your conclusion stands true: it might be faster to just do it manually. That's the state of LLMs as of today. Your post neither hints at such detailed and laborious prompting nor seem to recognize you've asked it too much, displaying that you are not very comfortable with the limitations of the tool. You're still exploring what it can and what it can't do. But that also implies you're yet not an expert.

2) The second takeaway that you're not yet as comfortable with the tools as you think you are is clearly context management. 2/3k locs of code are way too much. It's a massive amount of output to hope for good results (this also ties with the quality of the prompt, with the guidelines and code practices provided, etc, etc).

3) Neither 1 or 2 are criticisms of your conclusions or opinions, if anything, they are confirmations of your point that LLMs are not there. But what I disagree with is the rush into concluding that AI coding provides net 0 benefits out of your experience. That I don't share. Instead of settling on what it could do (help with planning, writing a spec file, writing unit tests, providing the more boilerplate-y part of the code) and use the LLM to reduce the friction (and thus provide a net benefit), you essentially asked it to replace you and found out the obvious: that LLMs cannot take care of non-trivial business logic yet, and even when they can the results are nowhere near being satisfactory. But that doesn't mean that AI-assisted coding is useless and the net benefit is 0, or negative, it only becomes so as the expectations on the tool are too big and the amount of information provided is either too small to return consistent results or too large for the context to be an issue.

replies(2): >>45093727 #>>45094719 #
3. throwaway346434 ◴[01 Sep 25 15:50 UTC] No.45093727[source]▶
This is a kind of nuts take; - Senior engineer - Uses tools for non trivial undertaking - Didn't find value in it

Your conclusion from that is "but they are doing it wrong", while also claiming they are saying things they didn't say (0 net benefits, useless, etc).

Do you see how that might undermine your point? That you feel they haven't take the time to understand the tools, but you didn't actually read what what wrote?

4. Rochus ◴[01 Sep 25 17:27 UTC] No.45094719[source]▶
I don't know where your confidence or assumptions come from. Do you work for Anthropic? My prompts for the code generators included an 1.2kLOC code file plus detailed instructions (as described elsewhere), with more details during the session. So I don't think your points apply.