A Note on Essential Complexity

There's a lot to like in this article, but I would go a step further with its meta-abstractions. At complexity 0, there is no software to be written. In essence, complexity and the software itself are closely related, akin to the relationship between compression and AI. The article's thesis is that minimizing complexity is what engineering is about. That means we're kind of minimizing the total amount of software we have to write and maintain. Makes sense, so far so good.

Next up is architecture. More architecture generally means more complexity in the short term and less in the long term. But that's assuming we know what the long term will look like, which we can often only predict. If we under-provision architecture, there's a good chance we'll get buried in technical debt later on (though by then we may have switched gigs.) Enter the usual Agile arguments favoring the short term.

Last but not least, path-dependency. I think this is where the best designers shine. You might not need all the architecture right away, but you do need to steer clear of dead-ends. These are the bad quick-design decisions that bite you in the rear when the architecture eventually gets refactored in. An extreme example might be the lack of a version field.

IMO, awesome designers are great at predicting where the architecture will be and at forward-provisioning it. They don't need to build the freeway through the middle of town. They'll just plan a park here and a parking lot there, guessing the right spots.

>An extreme example might be the lack of a version field.

Nothing a "IF version is NULL THEN version = 1" can't fix ;)

I like this, but I have trouble with relating architecture to complexity. The "generally" qualifier serves as an apt escape hatch but permit me to ignore that for a moment.

Architecture emerges as the holistic shape of the design, the pattern of patterns, the bones of the system. It's possible to have architecture that reduces complexity by reducing scope and forcing apt compartmentalization into reusable, composable bits. One could perhaps even argue that GoF's Design Patterns attempts to commoditize the language to achieve greater degree of simplicity... but let's not.

I have trouble distinguishing architecture from design here. Imagine a kitchen with many cabinets, all alike. Some cupboards open by pulling a handle, some open when you push the handle, others swing up, or you must unscrew the handle. This is horrible and unintuitive, and surprising. Good architecture like good design eschews surprises (as in shock) and provides intuition by weaving consistency through the system. You build an expectation that things behave a certain way, and they do.

I would posit that good architecture, like good design, does not introduce complexity, rather the complexity is present regardless, and they delete layers of uncertainty and facilitate the necessary clarity to bring the overall objective into focus.

Perhaps there is a complexity cost introduced by adhering to a design or pattern or architecture, but if it's not outweighed by the credit it provides, offsetting overall complexity, then it's bad.

Your observation about path-dependency is on the money most certainly.

I really like this metaphor of provisioning architecture and staying clear of dead ends. I'd say dead ends come not just from bad quick design decisions but also from wrong predictions (in the scenario you describe where it's hard to tell what the long term looks like ---which is most of the times).

> awesome designers are great at predicting where the architecture will be and at forward-provisioning it.

And this matches the No Silver Bullet conclusion.

Agreed, and well put. I should have emphasized the role of uncertainty in the architecture/complexity relationship.

In your example, you have a minimum of 4 cabinets. If you start out with N>=4, architecture and complexity are inversely related (to a point of course.) They are sometimes directly related when 1<N<4, and most often are when N=1.

We're predicting the future path of N. Adding a subsystem and a formal internal API when N=1 is being bullish on N, trading the present for the future and expecting a good rate of return.

>More architecture generally means more complexity in the short term and less in the long term.

I'm not convinced of the latter. More architecture = more complexity in both short and long term, everything else being equal.

That is, if you can meet the required feature set with LESS architecture, it will be simpler both short and long term, compared to more architecture (which, since it's not essential for meeting the feature set, it would be adding abstractions and YAGNI features).