←back to thread

The unreasonable effectiveness of an LLM agent loop with tool use

(sketch.dev)
435 points crawshaw | 2 comments | 15 May 25 19:33 UTC | HN request time: 0.397s | source
Show context
_bin_ ◴[15 May 25 20:01 UTC] No.43998743[source]▶
I've found sonnet-3.7 to be incredibly inconsistent. It can do very well but has a strong tendency to get off-track and run off and do weird things.

3.5 is better for this, ime. I hooked claude desktop up to an MCP server to fake claude-code less the extortionate pricing and it works decently. I've been trying to apply it for rust work; it's not great yet (still doesn't really seem to "understand" rust's concepts) but can do some stuff if you make it `cargo check` after each change and stop it if it doesn't.

I expect something like o3-high is the best out there (aider leaderboards support this) either alone or in combination with 4.1, but tbh that's out of my price range. And frankly, I can't mentally get past paying a very high price for an LLM response that may or may not be useful; it leaves me incredibly resentful as a customer that your model can fail the task, requiring multiple "re-rolls", and you're passing that marginal cost to me.

replies(3): >>43998797 #>>43999022 #>>43999599 #
agilebyte ◴[15 May 25 20:07 UTC] No.43998797[source]▶
I am avoiding the cost of API access by using the chat/ui instead, in my case Google Gemini 2.5 Pro with the high token window. Repomix a whole repo. Paste it in with a standard prompt saying "return full source" (it tends to not follow this instruction after a few back and forths) and then apply the result back on top of the repo (vibe coded https://github.com/radekstepan/apply-llm-changes to help me with that). Else yeah, $5 spent on Cline with Claude 3.7 and instead of fixing my tests, I end up with if/else statements in the source code to make the tests pass.
replies(4): >>43999036 #>>43999080 #>>43999160 #>>44013021 #
nico ◴[15 May 25 20:44 UTC] No.43999160[source]▶
Cool tool. What format does it expect from the model?

I’ve been looking for something that can take “bare diffs” (unified diffs without line numbers), from the clipboard and then apply them directly on a buffer (an open file in vscode)

None of the paste diff extension for vscode work, as they expect a full unified diff/patch

I also tried a google-developed patch tool, but also wasn’t very good at taking in the bare diffs, and def couldn’t do clipboard

replies(1): >>43999210 #
1. agilebyte ◴[15 May 25 20:49 UTC] No.43999210[source]▶
Markdown format with a comment saying what the file path is. So:

This is src/components/Foo.tsx

```tsx // code goes here ```

OR

```tsx // src/components/Foo.tsx // code goes here ```

These seem to work the best.

I tried diff syntax, but Gemini 2.5 just produced way too many bugs.

I also tried using regex and creating an AST of the markdown doc and going from there, but ultimately settled on calling gpt-4.1-mini-2025-04-14 with the beginning of the code block (```) and 3 lines before and 3 lines after the beginning of the code block. It's fast/cheap enough to work.

Though I still have to make edits sometimes. WIP.

replies(1): >>43999528 #
2. ◴[15 May 25 21:32 UTC] No.43999528[source]▶