←back to thread

Test-driven development with an LLM for fun and profit

(blog.yfzhou.fyi)
219 points crazylogger | 1 comments | 16 Jan 25 15:30 UTC | HN request time: 1.06s | source
Show context
xianshou ◴[16 Jan 25 17:59 UTC] No.42728570[source]▶
One trend I've noticed, framed as a logical deduction:

1. Coding assistants based on o1 and Sonnet are pretty great at coding with <50k context, but degrade rapidly beyond that.

2. Coding agents do massively better when they have a test-driven reward signal.

3. If a problem can be framed in a way that a coding agent can solve, that speeds up development at least 10x from the base case of human + assistant.

4. From (1)-(3), if you can get all the necessary context into 50k tokens and measure progress via tests, you can speed up development by 10x.

5. Therefore all new development should be microservices written from scratch and interacting via cleanly defined APIs.

Sure enough, I see HN projects evolving in that direction.

replies(12): >>42729039 #>>42729413 #>>42729713 #>>42729788 #>>42730016 #>>42730842 #>>42731468 #>>42733881 #>>42735489 #>>42736464 #>>42740025 #>>42747244 #
Arcuru ◴[16 Jan 25 18:33 UTC] No.42729039[source]▶
> 5. Therefore all new development should be microservices written from scratch and interacting via cleanly defined APIs.

Not necessarily. You can get the same benefits you described in (1)-(3) by using clearly defined modules in your codebase, they don't need to be separate microservices.

replies(4): >>42729689 #>>42737545 #>>42738887 #>>42744790 #
1. theptip ◴[17 Jan 25 15:46 UTC] No.42738887[source]▶
Yeah, I think monorepos will be better for LLMs. Easier to refactor module boundaries as context grows or requirements change.

But practices like stronger module boundaries, module docs, acceptance tests on internal dev-facing module APIs, etc are all things that will be much more valuable for LLM consumption. (And might make things more pleasant for humans too!)