Most active commenters
  • shakna(4)
  • alightsoul(3)
  • guappa(3)
  • flumpcakes(3)

←back to thread

Andrej Karpathy: Software in the era of AI [video]

(www.youtube.com)
1479 points sandslash | 12 comments | 19 Jun 25 00:33 UTC | HN request time: 4.169s | source | bottom
Show context
alightsoul ◴[19 Jun 25 06:20 UTC] No.44315969[source]▶
why does vibe coding still involve any code at all? why can't an AI directly control the registers of a computer processor and graphics card, controlling a computer directly? why can't it draw on the screen directly, connected directly to the rows and columns of an LCD screen? what if an AI agent was implemented in hardware, with a processor for AI, a normal computer processor for logic, and a processor that correlates UI elements to touches on the screen? and a network card, some RAM for temporary stuff like UI elements and some persistent storage for vectors that represent UI elements and past converstations
replies(4): >>44315999 #>>44316015 #>>44316024 #>>44316162 #
flumpcakes ◴[19 Jun 25 06:25 UTC] No.44315999[source]▶
I'm not sure this makes sense as a question. Registers are 'controlled' by running code for a given state. An AI can write code that changes registers, as all code does in operation. An AI can't directly 'control registers' in any other way, just as you or I can't.
replies(2): >>44316018 #>>44316020 #
1. alightsoul ◴[19 Jun 25 06:29 UTC] No.44316020[source]▶
I would like to make an AI agent that directly interfaces with a processor by setting bits in a processor register, thus eliminating the need for even assembly code or any kind of code. The only software you would ever need would be the AI.
replies(2): >>44316080 #>>44318818 #
2. shakna ◴[19 Jun 25 06:40 UTC] No.44316080[source]▶
That's called a JIT compiler. And ignoring how bad an idea blending those two... It wouldn't be that difficult a task.

The hardest parts of a jit is the safety aspect. And AI already violates most of that.

replies(2): >>44316117 #>>44318821 #
3. alightsoul ◴[19 Jun 25 06:48 UTC] No.44316117[source]▶
The safety part will probably be either solved or a non-issue or ignored. Similarly to how GPT3 was often seen as dangerous before ChatGPT was released. Some people who have only ever vibe coded are finding jobs today, ignoring safety entirely and lacking a notion of it or what it means. They just copy paste output from ChatGPT or an agentic IDE. To me it's JIT already with extra steps. Or they have pivoted their software engineers to vibe coding most of the time and don't even touch code anymore doing JIT with extra steps again.
replies(1): >>44316709 #
4. shakna ◴[19 Jun 25 08:51 UTC] No.44316709{3}[source]▶
As "jit" to you means running code, and not "building and executing machine code", maybe you could vibe code this. And enjoy the segfaults.
replies(1): >>44316939 #
5. guappa ◴[19 Jun 25 09:34 UTC] No.44316939{4}[source]▶
In a way he's making sense. If the "code" is the prompt, the output of the llm is an intermediate artifact, like the intermediate steps of gcc.

So why should we still need gcc?

The answer is of course, that we need it because llm's output is shit 90% of the time and debugging assembly or binary directly is even harder, so putting asides the difficulties of training the model, the output would be unusable.

replies(1): >>44317051 #
6. shakna ◴[19 Jun 25 09:56 UTC] No.44317051{5}[source]▶
Probably too much snark from me. But the gulf between interpreter and compiler can be decades of work, often discovering new mathematical principles along the way.

The idea that you're fine to risk everything, in the way agentic things allow [0], and want that messing around with raw memory is... A return to DOS' crashes, but with HAL along for the ride.

[0] https://msrc.microsoft.com/update-guide/vulnerability/CVE-20...

replies(1): >>44317216 #
7. guappa ◴[19 Jun 25 10:29 UTC] No.44317216{6}[source]▶
Ah don't worry, llms are a return to crashes as it is :)

The other day it managed to produce code that made python segfault.

replies(1): >>44326596 #
8. flumpcakes ◴[19 Jun 25 14:07 UTC] No.44318818[source]▶
This makes no sense at all. You can't set registers without assembly code. If you could set registers without assembly code then it would be pointless as the registers wouldn't be 'running' against anything.
9. flumpcakes ◴[19 Jun 25 14:08 UTC] No.44318821[source]▶
It's not a JIT. A JIT produces assembly. You can't "set registers" or do anything useful without assembly code running on the processor.
replies(1): >>44333180 #
10. diggan ◴[20 Jun 25 11:04 UTC] No.44326596{7}[source]▶
> produce code that made python segfault

To be fair, that's pretty easy for a human to do too.

replies(1): >>44354476 #
11. shakna ◴[20 Jun 25 23:45 UTC] No.44333180{3}[source]▶
Riiight... Which was my point? If you want an AI able to set registers, you want to hook it to a JIT. Which avoids assembly by setting machine code directly into memory and executing said memory.
12. guappa ◴[23 Jun 25 11:14 UTC] No.44354476{8}[source]▶
On purpose yes. But the entire point of languages with managed memory is that they do not segfault.