←back to thread

The Rise of Whatever

(eev.ee)
628 points cratermoon | 4 comments | 04 Jul 25 04:48 UTC | HN request time: 0.868s | source
Show context
tptacek ◴[04 Jul 25 05:29 UTC] No.44461381[source]▶
LLM output is crap. It’s just crap. It sucks, and is bad.

Still don't get it. LLM outputs are nondeterministic. LLMs invent APIs that don't exist. That's why you filter those outputs through agent constructions, which actually compile code. The nondeterminism of LLMs don't make your compiler nondeterministic.

All sorts of ways to knock LLM-generated code. Most I disagree with, all colorable. But this article is based on a model of LLM code generation from 6 months ago which is simply no longer true, and you can't gaslight your way back to Q1 2024.

replies(7): >>44461418 #>>44461426 #>>44461474 #>>44461544 #>>44461933 #>>44461994 #>>44463037 #
1. eviks ◴[04 Jul 25 07:22 UTC] No.44461994[source]▶
> or they suggested an elaborate and tedious workaround that would technically solve the problem (but introduce new ones).

There is no value in randomly choosing an API that exists. There is value in choosing an API that works.

When LLM makes up an API that doesn't even exist it indicates that it's not tied to the reality of the task of fetching a working API, so filtering out nonexistent APIs will not help the other results match better. But yes, they'll compile.

replies(1): >>44462082 #
2. tptacek ◴[04 Jul 25 07:34 UTC] No.44462082[source]▶
Give me a break. First, that's not the claim the article makes. Second, that's not the experience of anybody who actually uses Claude Code or Gemini Desktop or whatever people are using this week. This is what I'm talking about: people just gaslighting.

LLMs can write truly shitty code. I have a <50% hit rate on stuff I don't have to rewrite, with Sketch.dev, an agent I like a lot. But they don't fail the way you or this article claim they do. Enough.

replies(1): >>44462228 #
3. eviks ◴[04 Jul 25 07:56 UTC] No.44462228[source]▶
First, it is, you've just reduced the forest article claim down to a single tree to make it appear like your "solution" cuts it.

Second, speak for yourself, you have no clue about everybody's experience to make such a universal claim.

Lastly, the article talks about author's experience, not yours, so you're the only one who can gaslight the author, not the other way around

replies(1): >>44462236 #
4. tptacek ◴[04 Jul 25 07:57 UTC] No.44462236{3}[source]▶
I'm comfortable with the assertions I'm making here and stand by them.