A Note on Essential Complexity

Humans simply cannot keep the whole stack for a complex system in memory, that is why we abstract layers with APIs etc and generally specialize on one layer only.

My (Sci-Fi) book postulated that an AGI (a real AGI, after all it was Sci-Fi) would simply discard everything the humans wrote and rewrite the complete stack (including later on the Hardware and ISA) in machine code without anything unnecessary for the task and of course totally unreadable to a human. It is an interesting scenario to ponder.

OTOH a lot of the abstractions are created because problems aren't solved coherently but rather partitioned into organizational structures adding incidental complexity. The worst are <noun>-services that seem to always do too much/too little or have interface designs that suit itself than needs of clients/consumers. I spend a fair amount of time decomplexifying the org out of architectures when we realize that performance sucks because reasons.

We can do a lot of that discarding of excess by realizing that there's the data that's read, the rules applied, and the data that's written. Everything else is ephemeral and plumbing. Looking at a process that way, and seeing the amount of complexity/abstractions one can ask what all that really necessary? Does it pay for itself in some way? It has to justify its existence beyond implementing the needed process.

I question the value of frameworks sometimes[0].

[0] https://ptrthomas.wordpress.com/2006/06/06/java-call-stack-f...

> there's the data that's read, the rules applied, and the data that's written

and each rule applied can do one of three basic things: https://news.ycombinator.com/item?id=32498382

I like to collapse layers that are doing the "same kind" of thing according to the taxonomy above.