Most active commenters
  • skydhash(3)

←back to thread

75 points throwaway-ai-qs | 18 comments | | HN request time: 0.672s | source | bottom

Between code reviews, and AI generated rubbish, I've had it. Whether it's people relying on AI to write pull request descriptions (that are crap by the way), or using it to generate tests.. I'm sick of it.

Over the year, I've been doing a tonne of consulting. The last three months I've watched at least 8 companies embrace AI generated pip for coding, testing, and code reviews. Honestly, the best suggestions I've seen are found by linters in CI, and spell checkers. Is this what we've come to?

My question for my fellow HNers.. is this what the future holds? Is this everywhere? I think I'm finally ready to get off the ride.

1. nharada ◴[] No.45278990[source]
My biggest annoyance is that people aren't transparent about when they use AI, and thus you are forced to review everything through the lens that it may be human created and thus deserving of your attention and benefit of the doubt.

When an AI generates some nonsense I have zero problem changing or deleting it, but if it's human-written I have to be aware that I may be missing context/understanding and also cognizant of the author's feelings if I just re-write the entire thing without their input.

It's a huge amount of work offloaded on me, the reviewer.

replies(2): >>45279083 #>>45279169 #
2. kstrauser ◴[] No.45279083[source]
I disagree. Code is code: it speaks for itself. If it's high quality, I don't care whether it came from a human or an AI trained on good code examples. If it sucks, it's not somehow less awful just because someone worked really hard on it. What would change for me is how tactful I am in wording my response to it, in which case it's a little nicer replying to AIs because I don't care about being mean to them. The summary of my review would be the same either way: here are the bad parts I want you to re-work before I consider this.
replies(10): >>45279121 #>>45279166 #>>45279176 #>>45279188 #>>45279282 #>>45279301 #>>45279327 #>>45279336 #>>45279362 #>>45280127 #
3. jacknews ◴[] No.45279121[source]
Half of the point of code review is to provide expert or alternative feedback to junior and other developers, to share solutions and create a consistent style, standard and approach.

So no, code is not just code.

4. chomp ◴[] No.45279166[source]
> What would change for me is how tactful I am in wording my response to it

So code is not code? You’re admitting that provenance matters in how you handle it.

5. Workaccount2 ◴[] No.45279169[source]
>My biggest annoyance is that people aren't transparent about when they use AI

You get shamed and dismissed for mentioning that you used AI, so naturally nobody mentions they used AI. They mention AI the first time, see the blow back, and never mention it again. It just shows how myopic group-think can be.

6. alansammarone ◴[] No.45279176[source]
I've had a similar discussion with a coworker which I respect and know to be very experienced, and interestingly we disagreed on this very point. I'm with you - I think AI is just tool, and people shouldn't be off the hook because they used AI code. If they consistently deliver bad code, bad PR descriptions, or fail to explain/articulate their reasoning, I don't see any particular reason we should treat it any differently now that AI exists. It goes both ways, of course - reviewer also shouldn't pay less attention when the code is did not involve AI help in any form. I think these are completely orthogonal and I honestly don't see why people have this view.

The person who created the PR is responsible for it. Period. Nothing changes.

replies(1): >>45279314 #
7. roughly ◴[] No.45279188[source]
The problem with AI generated code is there’s no unifying theory behind the change. When a human writes code and one part of the system looks weird or different, there’s usually a reason - by digging in on the underlying system, I can usually figure out what they were going for or why they did something in a particular way. I can only provide useful feedback or alternatives if I can figure out why something was done, though.

LLM-generating code has no unifying theory behind it - every line may as well have been written by a different person, so you get an utterly insane looking codebase with no constant thread tying it together and no reason why. It’s like trying to figure out what the fuck is happening in a legacy codebase, except it’s brand new. I’ve wasted hours trying to understand someone’s MR, only to realize it’s vibe code and there’s no reason for any of it.

replies(1): >>45282486 #
8. skydhash ◴[] No.45279282[source]
Adding to the sibling comment by @jacknews. Code is much more than an algorithm, it’s the description of the algorithm that is non ambiguous and human readable. Code review is a communication tool. The basic expectation is that you’re a professional and I’m just adding another set of eyes, or you’re a junior and I’m reviewing for the sake of training.

So when there’s some confusion, I’m going back to the author. Because you should know why each line was written and how it contributes to the solution.

But a complete review takes time. So in a lot of places, we only do a quick scan checking for unusual stuff instead of truly reviewing the algorithms. That’s because we trust our colleagues to test and verify their own work. Which AI users usually skip.

9. jcranmer ◴[] No.45279301[source]
The problem with reviewing AI-written code is that AI makes mistakes in very different ways from the way humans make mistakes, so you essentially have to retrain yourself to watch for the kinds of mistakes that AI makes.
10. skydhash ◴[] No.45279314{3}[source]
It does because the amount of PR goes up. So instead of reviewing, it’s more like back and forth debugging where you are doing the check that the author was supposed to do.
replies(1): >>45279372 #
11. risyachka ◴[] No.45279327[source]
This is all great except it doesn't give any reason not to label AI code.
12. elviejo ◴[] No.45279336[source]
Code is not only code.

It's like saying physics it's just math. If we read:

F = m*a

There is ton of knowledge encoded in that formula.

We cannot evaluate the formula alone. We need the knowledge behind it to see if it matches reality.

With llms we know for a fact that if the code matches reality, or expectations, it's a happy accident.

13. bluGill ◴[] No.45279362[source]
Code doesn't always speak for itself. I've had to do some weird things that make no sense on its own. I leave comments but they are not always easy to understand. Most of this is when I'm sharing data across threads - there is good reason for each lock/atomic and each missing lock. (I avoid writing such code, but sometimes there is no choice). If AI is writing such code I don't trust them to figure out those details, while I have some (only a minority but some) coworkers I trust to figure this out.
14. alansammarone ◴[] No.45279372{4}[source]
So the author is not a great programmer/professional. I agree with you that they should have done their homework, tested it, have a mental model for why and how, etc. If they don't, it doesn't seem to be particularly relevant to me if that's because they had a concussion or because they use AI.
replies(1): >>45279552 #
15. skydhash ◴[] No.45279552{5}[source]
It’s easy to skip quality with code, starting with coding only the happy path and bad design that hides bugs. Handling errors properly can take a lot of times, and designing to avoid errors takes longer.

So when you have a tools that can produce things that fits the happy path easily, don’t be surprised that the amount of PRs goes up. Because before, by the time you can write the happy path that easily, experience has taught you all the error cases that you would have skipped.

replies(1): >>45282467 #
16. m463 ◴[] No.45280127[source]
> Code is code

oh come on.

That's like saying "food is food" or "an AI howto is the same as a human-written howto".

The problem is that code that looks good is not the same as code that is good, but they are superficially similar to a reviewer.

and... you can absolutely bury reviewers in it.

17. JustExAWS ◴[] No.45282467{6}[source]
I have been developing for 40 years as a hobbyist for 10 and a professional for 30. I always started with the happy path and made it sure it worked and then kept thinking about corner cases. If an LLM can get me through the happy path (and it often generates code to guard against corner cases) why wouldn’t I use it?
18. scarface_74 ◴[] No.45282486{3}[source]
How is that different then long lived code bases where various developers with varying skill levels have modified it?