←back to thread

A staff engineer's journey with Claude Code

(www.sanity.io)
548 points kmelve | 2 comments | 02 Sep 25 19:34 UTC | HN request time: 0.467s | source
Show context
spicyusername ◴[03 Sep 25 11:41 UTC] No.45114584[source]▶
I guess we're just going to be in the age of this conversation topic until everyone gets tired of talking about it.

Every one of these discussions boils down to the following:

- LLMs are not good at writing code on their own unless it's extremely simple or boilerplate

- LLMs can be good at helping you debug existing code

- LLMs can be good at brainstorming solutions to new problems

- The code that is written by LLMs always needs to be heavily monitored for correctness, style, and design, and then typically edited down, often to at least half its original size

- LLMs utility is high enough that it is now going to be a standard tool in the toolbox of every software engineer, but it is definitely not replacing anyone at current capability.

- New software engineers are going to suffer the most because they know how to edit the responses the least, but this was true when they wrote their own code with stack overflow.

- At senior level, sometimes using LLMs is going to save you a ton of time and sometimes it's going to waste your time. Net-net, it's probably positive, but there are definitely some horrible days where you spend too long going back and forth, when you should have just tried to solve the problem yourself.

replies(12): >>45114610 #>>45114779 #>>45114830 #>>45115041 #>>45115537 #>>45115567 #>>45115676 #>>45115681 #>>45116405 #>>45116622 #>>45118918 #>>45120482 #
1. theshrike79 ◴[03 Sep 25 13:40 UTC] No.45115681[source]▶
> - The code that is written by LLMs always needs to be heavily monitored for correctness, style, and design, and then typically edited down, often to at least half its original size

For this language matters a lot, if whatever you're using has robust tools for linting and style checks, it makes the LLMs job a lot easier. Give it a rule (or a forced hook) to always run tests and linters before claiming a job is done and it'll iterate until what it produces matches the rules.

But LLM code has a habit of being very verbose and covers every situation no matter how minuscule.

This is especially grating when you're doing a simple project for local use and it's bootstrapping something that's enterprise-ready :D

replies(1): >>45116604 #
2. WorldMaker ◴[03 Sep 25 14:59 UTC] No.45116604[source]▶
If you force the LLM to solve every test failure this also can lead to the same breakdown models as very junior developers coding to the tests rather than the problems, I've seen all of:

1) I broke the tests, guess I should delete them.

2) I broke the tests, guess the code I wrote was wrong, guess I should delete all of that code I wrote.

3) I broke the tests, guess I should keep adding more code and scaffolding. Another abstraction layer might work? What if I just add skeleton code randomly, does this add random code whack-a-mole work?

That last one can be particularly "fun" because already verbose LLM code skyrockets into baroque million line PRs when left truly unsupervised, and that PR still won't build or pass tests.

There's no true understanding by an LLM. Forcing it to lint/build can be important/useful, but still not a cure-all, and leads to such fun even more degenerate cases than hand-holding it.