←back to thread

728 points freetonik | 10 comments | | HN request time: 0s | source | bottom
Show context
Waterluvian ◴[] No.44976790[source]
I’m not a big AI fan but I do see it as just another tool in your toolbox. I wouldn’t really care how someone got to the end result that is a PR.

But I also think that if a maintainer asks you to jump before submitting a PR, you politely ask, “how high?”

replies(16): >>44976860 #>>44976869 #>>44976945 #>>44977015 #>>44977025 #>>44977121 #>>44977142 #>>44977241 #>>44977503 #>>44978050 #>>44978116 #>>44978159 #>>44978240 #>>44978311 #>>44978533 #>>44979437 #
cvoss ◴[] No.44976945[source]
It does matter how and where a PR comes from, because reviewers are fallible and finite, so trust enters the equation inevitably. You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.

If trust didn't matter, there wouldn't have been a need for the Linux Kernel team to ban the University of Minnesota for attempting to intentionally smuggle bugs through the PR process as part of an unauthorized social experiment. As it stands, if you / your PRs can't be trusted, they should not even be admitted to the review process.

replies(4): >>44977169 #>>44977263 #>>44978862 #>>44979553 #
koolba ◴[] No.44977169[source]
> You must ask "Do I trust where this came from?" And to answer that, you need to know where it come from.

No you don’t. You can’t outsource trust determinations. Especially to the people you claim not to trust!

You make the judgement call by looking at the code and your known history of the contributor.

Nobody cares if contributors use an LLM or a magnetic needle to generate code. They care if bad code gets introduced or bad patches waste reviewers’ time.

replies(3): >>44977245 #>>44977531 #>>44978479 #
geraneum ◴[] No.44977531[source]
> Nobody cares if contributors use an LLM or a magnetic needle to generate code.

That’s exactly opposite of what the author is saying. He mentions that [if the code is not good, or you are a beginner] he will help you get to finish line, but if it’s LLM code, he shouldn’t be putting effort because there’s no human on the other side.

It makes sense to me.

replies(1): >>44978344 #
macawfish ◴[] No.44978344[source]
"but if it’s LLM code, he shouldn’t be putting effort because there’s no human on the other side"

That's the false equivalence right there

replies(1): >>44978651 #
1. tsimionescu ◴[] No.44978651{3}[source]
It's not a false equivalence. You can teach a beginner to become an intermediate (and later a master, if they stick to it). You can't teach an LLM to be better. Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota.
replies(1): >>44978685 #
2. macawfish ◴[] No.44978685[source]
"Every piece of feedback you give to an LLM is like screaming into the void - it wastes your time, and doesn't change the LLM one iota."

I think you just haven't gotten the hang of it yet, which is fine... the tooling is very immature and hard to get consistent results with. But this isn't a given. Some people do get good, steerable LLM coding setups.

replies(2): >>44979402 #>>44979604 #
3. sho_hn ◴[] No.44979402[source]
Steering via prompting isn't the same as fundamentally changing the LLM by teaching, as you can do with humans. I think OP understands this better than you.
replies(1): >>44979589 #
4. macawfish ◴[] No.44979589{3}[source]
Can't tell if you're responding in earnest or not here?

LLMs are trained to be steerable at inference time via context/prompting. Fine tuning is also possible and often used. Both count as "feedback" in my book, and my point is that both can be effective at "changing the LLM" in terms of its behavior at inference time.

replies(1): >>44980193 #
5. david_allison ◴[] No.44979604[source]
As a maintainer, if you're dealing with a contributor who's sending in AI slop, you have no opportunity to prompt the LLM.

The PR effectively ends up being an extremely high-latency conversation with an LLM, via another human who doesn't have the full context/understanding of the problem.

replies(1): >>44980772 #
6. sho_hn ◴[] No.44980193{4}[source]
And also clearly not what the OP means, who was trying to make a point that tuning the prompt to an otherwise stateless LLM inference job is nothing at all like teaching a human being. Mechanically, computationally, morally or emotionally. For example, humans aren't just tools; giving feedback to LLMs does little to further their agency.
replies(1): >>44980767 #
7. macawfish ◴[] No.44980767{5}[source]
The false equivalence I pointed at earlier was "LLM code => no human on the other side".

The person driving the LLM is a teachable human who can learn what's what's going on and learn to improve the code. It's simply not true that there's no person on the other side of the PR.

The idea that we should be comparing "teaching a human" to "teaching an LLM" is yet another instance of this false equivalence.

It's not inherently pointless to provide feedback on a PR with code written using an LLM, that feedback goes to the person using the LLM tools.

People are swallowing this b.s. marketing mystification of "LLMs as non human entities". But really they're fancy compilers that we have a lot to learn about.

replies(1): >>44981579 #
8. macawfish ◴[] No.44980772{3}[source]
You're totally dismissing this person's agency and their ability to learn. You're all but writing off their existence.
9. nullc ◴[] No.44981579{6}[source]
The person operating the LLM is not a meaningfully teachable human when they're not disclosing that they're using an LLM.

IF they disclose what they've done, provided the prompts, etc. then other contributors can help them get better results from the tools. But the feedback is very different than the feedback you'd give a human that actually wrote the code in question, that latter feedback is unlikely to be of much value (and even less likely to persist).

replies(1): >>44982704 #
10. sho_hn ◴[] No.44982704{7}[source]
Yep, true.

I've done things like share a ChatGPT account with a junior dev to steer them toward better prompts, actually, and that had some merit.