←back to thread

Living Dangerously with Claude

(simonwillison.net)
149 points FromTheArchives | 2 comments | 22 Oct 25 12:36 UTC | HN request time: 0s | source
Show context
matthewdgreen ◴[23 Oct 25 01:10 UTC] No.45677089[source]▶
So let me get this straight. You’re writing tens of thousands of lines of code that will presumably go into a public GitHub repository and/or be served from some location. Even if it only runs locally on your own machine, at some point you’ll presumably give that code network access. And that code is being developed (without much review) by an agent that, in our threat model, has been fully subverted by prompt injection?

Sandboxing the agent hardly seems like a sufficient defense here.

replies(3): >>45677537 #>>45684527 #>>45686450 #
tptacek ◴[23 Oct 25 17:34 UTC] No.45684527[source]▶
Where did "without much review" come from? I don't see that in the deck.
replies(2): >>45684731 #>>45688191 #
1. matthewdgreen ◴[23 Oct 25 22:29 UTC] No.45688191{3}[source]▶
He wrote 14,000 lines of code in several days. How much review is going on there?
replies(1): >>45688711 #
2. simonw ◴[23 Oct 25 23:20 UTC] No.45688711[source]▶
Oh hang on, I think I've spotted a point of confusion here.

All three of the projects I described in this talk have effectively zero risk in terms of containing harmful unreviewed code.

DeepSeek-OCR on the Spark? I ran that one in a Docker container, saved some notes on the process and then literally threw away the container once it had finished.

The Pyodide in Node.js one I did actually review, because its code I execute on a machine that isn't disposable. The initial research ran in a disposable remote container though (Claude Code for web).

The Perl in WebAssembly one? That runs in a browser sandbox. There's effectively nothing bad that can happen there, that's why I like WebAssembly so much.

I am a whole lot more cautious in reviewing code that has real stakes attached to it.