Andrej Karpathy: Software in the era of AI [video]

Meanwhile, I asked this morning Claude 4 to write a simple EXIF normalizer. After two rounds of prompting it to double-check its code, I still had to point out that it makes no sense to load the entire image for re-orientating if the EXIF orientation is fine in the first place.

Vibe vs reality, and anyone actually working in the space daily can attest how brittle these systems are.

Maybe this changes in SWE with more automated tests in verifiable simulators, but the real world is far to complex to simulate in its vastness.

The real question is how long it'll take until they're not brittle

“Treat it like a junior developer” … 5 years later … “Treat it like a junior developer”

Usable LLMs are 3 years old at this point. ChatGPT, not Github Copilot, is the marker.