←back to thread

214 points Brajeshwar | 5 comments | | HN request time: 0s | source
Show context
Rochus ◴[] No.45090991[source]
The article claims, that senior developers with over 10 years of experience are more than twice as likely to heavily rely on AI tools compared to their junior counterparts. No p-values or statistical significance tests are reported in either The Register article or Fastly's original blog post.

I have over 30 years of experience and recently used Claude Opus 4.1 (via browser and claude.ai) to generate an ECMA-335 and an LLVM code generator for a compiler, and a Qt adapter for the Mono soft debugging protocol. Each task resulted in 2-3kLOC of C++.

The Claude experience was mixed; there is a high probability that the system doesn't respond or just quickly shows an overloaded message and does nothing. If it generates code, I quckly run in some output limitation and have to manually press "continue", and then often the result gets scrambled (i.e. the order of the generated code fragments gets mixed up, which requires another round with Claude to fix).

After this process, the resulting code then compiled immediately, which impressed me. But it is full of omissions and logical errors. I am still testing and correcting. All in all, I can't say at this point that Claude has really taken any work off my hands. In order to understand the code and assess the correctness of the intermediate results, I need to know exactly how to implement the problem myself. And you have to test everything in detail and do a lot of redesigning and correcting. Some implementations are just stubs, and even after several attempts, there was still no implementation.

In my opinion, what is currently available (via my $20 subscription) is impressive, but it neither replaces experience nor does it really save time.

So yes, now I'm one of the 30% seniors who used AI tools, but I didn't really benefit from them in these specific tasks. Not surprisingly, also the original blog states, that nearly 30% of senior developers report "editing AI output enough to offset most of the time savings". So not really a success so far. But all in all I'm still impressed.

replies(5): >>45091343 #>>45091757 #>>45092344 #>>45092985 #>>45099223 #
oliwary ◴[] No.45091343[source]
Hey! I would encourage you to try our Claude code instead, which is also part of your subscription. It's a CLI that takes care of many of the issues you encountered, as it works directly on the code files in a directory. No more copy pasting or unscrambling results. Likewise, it can run commands itself to e.g. compile or even test code.
replies(3): >>45091480 #>>45091506 #>>45091691 #
Rochus ◴[] No.45091480[source]
I'm working on old hardware and not-recent Linux and compiler versions, and I have no confidence yet in allowing AI direct (write) access to my repositories.

Instead I provided Claude with the source code of a transpiler to C (one file) which is known to work, uses the same IR as the new code generators were supposed to use.

This is a controlled experiment with a clear and complete input and clear expectations and specifications of the output. I don't think I would be able to clearly isolate the contributions and assess the performance of Claude when it has access to arbitrary parts of the source code.

replies(4): >>45091766 #>>45091898 #>>45091902 #>>45092832 #
stavros ◴[] No.45091902[source]
I use Claude Code with the Max plan, and the experience isn't far off from what you describe. You still need to understand the system and review the implementation, because it makes many mistakes.

That's not the part it saves me time in, it saves me time in looking up the documentation. Other than that, it might be slower, because the larger the code change is, the more time I need to spend reviewing, and past a point I just can't be bothered.

The best way I've found is to have it write small functions, and then I tell it to compose them together. That way, I know exactly what's happening in the code, and I can trust that it works correctly. Cursor is probably a better way to do that than Claude Code, though.

replies(3): >>45092089 #>>45092284 #>>45092326 #
1. Rochus ◴[] No.45092284[source]
> it saves me time in looking up the documentation

I have a Perplexity subscription which I heavily use for such purpose, just asking how something works or should be used, with a response just on the point and with examples. Very useful indeed. Perplexity gives me access to Claude Sonnet 4 w/o Thinking which I consider great models, and it can also generate decent code. My intention was to find out how good the recent Claude Opus is in comparison and how much of my work I'm able to delegate. Personally I much prefer the user interface features, performance and availability of Perplexity to Claude.ai.

replies(2): >>45092757 #>>45092844 #
2. stavros ◴[] No.45092757[source]
I have both, and Perplexity is much more like a search engine than a chat companion (or at least that's how I use it). I like both, though.
replies(1): >>45095598 #
3. gommm ◴[] No.45092844[source]
I end up using Perplexity a lot too, especially when I'm doing something unfamiliar. It's also a good way to quickly find out what are best practices for a given framework/language I'm not that familiar with (I usually ask it to link to examples in the wild and it find opensource projects illustrating those points)
4. Rochus ◴[] No.45095598[source]
You can select the model. I very much appreciate the Claude Sonnet models which are very good and rational discussion partners, responding to arguments in detail and critically, allowing for the dialectical exploration of complex topics. I have also experimented with other models including ChatGPT, Gemini or Grok, but the resulting discussions were only a fraction as useful (i.e. more optimized towards affirmative feel-good small talk, from my humble point of view).
replies(1): >>45096122 #
5. stavros ◴[] No.45096122{3}[source]
Hmm, I've never tried that, even though I prefer Claude in general too. I'll try that, thanks!