←back to thread

Naur's "Programming as Theory Building" and LLMs replacing human programmers

(ratfactor.com)
108 points bertman | 1 comments | 28 Apr 25 06:16 UTC | HN request time: 0.253s | source
Show context
falcor84 ◴[28 Apr 25 13:25 UTC] No.43821300[source]▶
> First, you cannot obtain the "theory" of a large program without actually working with that program...

> Second, you cannot effectively work on a large program without a working "theory" of that program...

I find the whole argument and particularly the above to be a senseless rejection of bootstrapping. Obviously there was a point in time (for any program, individual programmer and humanity as a whole) that we didn't have a "theory" and didn't do the work, but now we have both, so a program and its theory can appear "de novo".

So with that in mind, how can we reject the possibility that as an AI Agent (e.g. Aider) works on a program over time, it bootstraps a theory?

replies(4): >>43821340 #>>43821987 #>>43822329 #>>43822492 #
1. mlsu ◴[28 Apr 25 15:23 UTC] No.43822492[source]▶
The information needs to propagate through the network either forward (when the model has the codebase in context) or backward (when it updates its weights).

You can have the models pseudo “learn” by putting things in something like a system prompt but this is limited by context, and they will never permanently learn. But we don’t train at inference time with today’s LLMs.

We can explicitly reject this possibility by looking at the information that goes into the model at train and test time.