←back to thread

Context is the bottleneck for coding agents now

(runnercode.com)
196 points zmccormick7 | 3 comments | 26 Sep 25 15:06 UTC | HN request time: 0.709s | source
Show context
aliljet ◴[26 Sep 25 15:27 UTC] No.45387614[source]▶
There's a misunderstanding here broadly. Context could be infinite, but the real bottleneck is understanding intent late in a multi-step operation. A human can effectively discard or disregard prior information as the narrow window of focus moves to a new task, LLMs seem incredibly bad at this.

Having more context, but leaving open an inability to effectively focus on the latest task is the real problem.

replies(10): >>45387639 #>>45387672 #>>45387700 #>>45387992 #>>45388228 #>>45388271 #>>45388664 #>>45388965 #>>45389266 #>>45404093 #
tptacek ◴[26 Sep 25 16:25 UTC] No.45388271[source]▶
Asking, not arguing, but: why can't they? You can give an agent access to its own context and ask it to lobotomize itself like Eternal Sunshine. I just did that with a log ingestion agent (broad search to get the lay of the land, which eats a huge chunk of the context window, then narrow searches for weird stuff it spots, then go back and zap the big log search). I assume this is a normal approach, since someone else suggested it to me.
replies(2): >>45388348 #>>45388456 #
simonw ◴[26 Sep 25 16:31 UTC] No.45388348[source]▶
This is also the idea behind sub-agents. Claude Code answers questions about things like "where is the code that does X" by firing up a separate LLM running in a fresh context, posing it the question and having it answer back when it finds the answer. https://simonwillison.net/2025/Jun/2/claude-trace/
replies(2): >>45388378 #>>45388417 #
tra3 ◴[26 Sep 25 16:38 UTC] No.45388417[source]▶
I keep wondering if we're forgetting the fundamentals:

> Everyone knows that debugging is twice as hard as writing a program in the first place. So if you’re as clever as you can be when you write it, how will you ever debug it?

https://www.laws-of-software.com/laws/kernighan/

Sure, you eat the elephant one bite at a time, and recursion is a thing but I wonder where the tipping point here is.

replies(1): >>45388460 #
1. tptacek ◴[26 Sep 25 16:41 UTC] No.45388460[source]▶
I think recursion is the wrong way to look at this, for what it's worth.
replies(1): >>45388565 #
2. tra3 ◴[26 Sep 25 16:51 UTC] No.45388565[source]▶
Recursion and memoization only as a general approach to solving "large" problems.

I really want to paraphrase kernighan's law as applied to LLMs. "If you use your whole context window to code a solution to a problem, how are you going to debug it?".

replies(1): >>45388913 #
3. tptacek ◴[26 Sep 25 17:28 UTC] No.45388913[source]▶
By checkpointing once the agent loop has decided it's ready to hand off a solution, generating a structured summary of all the prior elements in the context, writing that to a file, and then marking all those prior context elements as dead so they don't occupy context window space.

Look carefully at a context window after solving a large problem, and I think in most cases you'll see even the 90th percentile token --- to say nothing of the median --- isn't valuable.

However large we're allowing frontier model context windows to get, we've got integer multiple more semantic space to allocate if we're even just a little bit smart about managing that resource. And again, this is assuming you don't recurse or divide the problem into multiple context windows.