←back to thread

Context is the bottleneck for coding agents now

(runnercode.com)
196 points zmccormick7 | 3 comments | 26 Sep 25 15:06 UTC | HN request time: 0s | source
Show context
asdev ◴[26 Sep 25 15:41 UTC] No.45387765[source]▶
I don't think intelligence is increasing. Arbitrary benchmarks don't reflect real world usage. Even with all the context it could possibly have, these models still miss/hallucinate things. Doesn't make them useless, but saying context is the bottleneck is incorrect.
replies(3): >>45388096 #>>45388362 #>>45398947 #
reclusive-sky ◴[26 Sep 25 16:09 UTC] No.45388096[source]▶
I agree, I often see Opus 4.1 and GPT5 (Thinking) make astoundingly stupid decisions with full confidence, even on trivial tasks requiring minimal context. Assuming they would make better decisions "if only they had more context" is a fallacy
replies(1): >>45388358 #
alchemist1e9 ◴[26 Sep 25 16:33 UTC] No.45388358[source]▶
Is there a good example you could provide of that? I just haven’t seen that personally so I’d be interested in any examples on these current models. I’m sure we all remember in the early days lots of examples of stupidity being posted and it was interesting. It be great if people kept doing that so we could get a better sense of which types of problems they are failing with astounding levels of stupidity on.
replies(2): >>45391205 #>>45395031 #
1. theshrike79 ◴[27 Sep 25 12:04 UTC] No.45395031[source]▶
I've had every single LLM I tried (Opus, Sonnet, GPT-5-(codex) and Grok light) all tell me that Go embeds[0] support relative paths UPWARDS in the tree.

They all have a very specific misunderstanding. Go embeds _do_ support relative paths like:

//go:embed files/hello.txt

But they DO NOT support any paths with ".." in it

//go:embed ../files/hello.txt

is not correct.

All confidently claimed that .. is correct and will work and tried to make it work multipled different ways until I pointed each to the documentation.

[0] https://pkg.go.dev/embed

replies(1): >>45395857 #
2. alchemist1e9 ◴[27 Sep 25 14:10 UTC] No.45395857[source]▶
I don’t really find that so surprising or particularly stupid. I was hoping to learn about serious issues with bad logic or reasoning not missing dots on i’s type stuff.

I can’t remember the example but there was another frequent hallucination that people were submitting bug reports that it wasn’t working, so the project looked at it and realized well actually that kinda would make sense and maybe our tool should work like that, and changed the code to work just like the LLM hallucination expected!

Also in general remember human developers hallucinate ALL THE TIME and then realize it or check documentation. So my point is I feel hallucinations are not particularly important or bother me as much as flawed reasoning.

replies(1): >>45410671 #
3. theshrike79 ◴[29 Sep 25 05:49 UTC] No.45410671[source]▶
Yep, LLMs are "just" statistical guessing machine.

And if an LLM guesses (hallucinates) a specific method for your API, it really should have it - statistically speaking =)