Basically, you should aim to minimise complexity in software design, but importantly, complexity is defined as "how difficult is it to make changes to it". "How difficult" is largely determined by the amount of cognitive load necessary to understand it.
Maybe I should make it more visible.
I even tried calling the bookstore on his campus and they said try back at the beginning of a semester, they didn’t have any copies.
My local book store could not source me a copy, and neither IIRC could Powell’s.
When you’re dealing with perennial plants, there’s only so much control you actually have, and there’s a list of things you know you have to do with them but you cannot do them all at once. There is what you need to do now, what you need to do next year, and a theory of what you’ll do over the next five years. And two years into any five year plan, the five year plan has completely changed. You’re hedging your bets.
Traditional Formal English and French gardens try to “master” the plants. Force them to behave to an exacting standard. It’s only possible with both a high degree of skill and a vast pool of labor. They aren’t really about nature, or food. They’re displays of opulence. They are conspicuous consumption. They are keeping up with the Joneses. Some people love that about them. More practical people see it as pretentious bullshit.
I think we all know a few companies that make a bad idea work by sheer force of will and overwhelming resources.
This is unhelpful even if the design is a complete mess.
I think it's a pretty good compromise. I have tried in the past not to duplicate code at all, and it often ends up more pain than gain. Allow copy/paste if code is needed in two different places, but refactor if needed in three or more, is a pretty good rule of thumb.
To me, it's the things that are specifically intended to behave the same should be kept DRY.
Deliberately going earlier makes sense if experience teaches you there will be 3+ of this eventually, but the point where I'm going to pick "Decline" and write that you need to fix this first is when I see you've repeated something 4-5 times, that's too many, we have machines to do repetition for us, have the machine do it.
An EnableEditor function? OK, meaningful name. EnablePublisher? Hmm, yes I understand the name scheme but I get a bad feeling. EnableCoAuthor? Approved with a stern note to reconsider, are we really never adding more of these, is there really some reason you can't factor this out? EnableAuditor. No. Stop, this function is named Enable and it takes a Role, do not copy-paste and change the names.
If you're copying and pasting something, there probably isn't a good reason for that. (The best common reason I can think of is "the language / framework demands so much boilerplate to reuse this little bit of code that it's a net loss" — which is still a bad feeling.)
If you rewrite something without noticing that you're doing so, something has definitely gone wrong.
If a client's requirements change to the point where you can't accommodate them in the nicely refactored function (or to the point where doing so would create an abomination) — then you can make the separate, similar looking version.
You can't win architecture arguments.
I like the article but the people who need it won't understand it and the people who don't need it already know this. As we say, it's not a technical problem, it's always a people and culture problem. Architecture just follows people and culture. If you have Rob Pike and Google you'll get Go. You can't read some book and make Go. (whether you like it or not is a different question).
Over time I have come to prefer having two near copies that are each more concretely expressive of their task than a more abstract version that caters to both.
Even the most obvious of functions like sin() and cos() may in some circumstances warrant a specialized implementation. Sure, for most stuff you should not have 10 copies of those all over the place. But sometimes you might.
DRY is a bad rule. The more appropriate rule is avoid duplicating code when not doing so results something better. I.e. judgement always trumps rules.
I would embrace copying and pasting for functionality that I want to be identical in two places right now, but I’m not sure ought to be identical in the future.
Sometimes things are only the same temporarily and shouldn't be brought together.
Ultimately, context, industries and teams vary so greatly that it doesn't make sense to quantify it.
What I've settled on instead is aiming for a balance between "mess" and "beauty" in my design. The hardest thing for me personally to grasp was that businesses are indeterministic whereas software is not, thus requirements always shifts and fitting this into the rigidity of computer systems is _difficult_.
These days, I only attempt to refactor when I start to feel the pain when I'm about to change the code.. and even then, I perform the bare minimum to clean up the code. Eventually multiple refactoring shapes a new pattern which can be pulled into an abstraction.
If you choose to not copy paste the code you better be damn sure the two places that use it are relying on the same concept, not just superficially similar code thats yet to diverge
There's no one rule. It takes experience and taste to make good guesses, and you'll often be wrong even so.
That's true. One doesn't change his mindset just after reading. Even after some mentorship the results are far from satisfying. Engineers can completely agree with you on the topic, only to go and do just the opposite.
It seems like the hardest thing to do is to build a feedback loop - "what decisions I made in past -> what it led to". Usually that loop takes a few years to complete, and most people forget that their architecture decisions led to a disaster. Or they just disassociate themselves.
Example: you have a config defined as Java/Go classes/structures. You want to check that the config file has the correct syntax. Non-DRY strategy is to describe its structure in an XSD schema (ok, ok JSON schema) and then validate the config. So you end up with two sources of truth: the schema and Java/Go classes, they can drift apart and cause problems.
The DRY way is to generate the classes/structures that define the config from that schema.
Now there’s a new requirement that only applies to Europe and nowhere else and it’s super easy and straight forward to change the infrastructure.
I don’t see how it was a poor choice to literally copy and paste configs that result in hundreds of thousands of lines of yaml and I have 25 yoe.
I remember using TCL in the 90s for my own projects as an embeddable command language. The main selling point was that it was a relatively widely understood scripting language with an easily embeddable off-the-shelf open source code base, perhaps one of the first of its kind (ignoring lisps.) Of course the limitations soon became clear. Only a few years later I had high hopes that Python would become a successor, but it went in a different direction and became significantly more difficult to embed in other applications than was TCL -- it just wasn't a primary use case for the core Python project. The modern TCL-equivalent is Lua, definitely a step up from TCL, but I think if EDA tools used Lua there would be plenty of hand-wringing too.
Just guessing, but I imagine that at the time TCL was adopted within EDA tools there were few alternatives. And once TCL was established it was going to be very hard to replace. Even if you ignore inertia at the EDA vendors, I can't imagine hardware engineers (or anyone with a job to do) wanting to switch languages every two to five years like some developers seem happy to do. It's a hard sell all around.
I reckon the best you can do is blame the vendors for (a) not choosing a more fit-for purpose language at the outset, which probably means Scheme, or inventing their own, (b) or not ripping the bandaid off at some point and switching to a more fit-for-purpose language. Blaming (b) is tough though, even today selecting an embedded application language is vexed: you want something that has good affordances as a language, is widely used and documented, easily embedded, and long-term stable. Almost everything I can think of fails the long term stability test (Python, JavaScript, even Lua which does not maintain backward compatibility between releases).
Downsides with your approach is:
1. Now whenever you want to change something both in Europe and (assuming) USA you have to do it in 2 places. If the change is the same for both, in my system, you could just update the default/shared config. If the change is different for both it's equally easy, but faster, since the overrides are smaller files.
2. It's not clear what the difference is between Europe and USA if there is 1 line different amongst thousands. If there are more differences in the future, it becomes increasingly difficult to tell the difference easily.
3. If in the future you also need to add Africa, you just compounded the problems of 1. and 2.
Sure, we could take the Foo, Bar, and Baz tables that share 80-90% of common logic and have them inherit from a common, shared, abstract component. We've discussed it in the past. Maybe it's the better solution, maybe not. But it would mean that instead of maintaining 3 component files and 3 test file, which are very similar, and when we need to change something it is often a copy-paste job, instead we'd have to maintain 2 additional files for the shared component, and when that has to change, it would require more work as we then have to add more to the other 3 files.
Such setups can often cause a cascade of tests that need updated and PRs with dozens of files changed.
Also, there are many parts of our project where things could be done much better if we were making them from scratch. But, 6 years of changing requirements and new features and this is what we have - and at this point, I'm not sure that having a shared component would actually make things easier unless we rewrite a huge amount of the codebase, for which there is no business reason.
https://www.youtube.com/watch?v=8bZh5LMaSmE
Worth watching in its entirety, but the quote is from ~13:59 in that video.
Unless it's a rule prohibiting complexity by removing technologies. Here's a set of rules I have in my head.
1. No multithreading. (See Mozilla's "You must be this high" sign)
2. No visitor pattern. (See grug oriented development)
3. No observer pattern. (See django when signals need to run in a particular order)
4. No custom DSL's. (I need to add a new operator, damnit, and I can't parse your badly written LALR(1) schema).
5. No XML. (Fight me, I have battle scars.)
That's not true. There's plenty of beginner programmers who will benefit from this.
But I'm not experienced enough to tell, whether it is my inexperience that causes the difficulty, or it is indeed that the unnecessary complexity tha causes it.
Somewhat off-topic, that's one usual failure mode of "DRY" code. Code is de-duplicated at a visual level rather than in terms of relevant semantics, so that changes which should only affect one path either affect both or are very complicated to reason about because of the unnecessary coupling.
This one is my particular pet-peeve. But I often think that the reason is because I suck. I'm going to read "grug".
I also hate one-liner functions.
From an operational perspective it’s much more important to ensure the code is clear and readable during an incident.
Overrides are like inheritance. They are themselves complex and add unnecessary cognitive load.
Composition is better for the common pieces that never change across regions. Think of an import statement of a common package into both the Europe and North America folders.
I easily see the one line diff among hundreds of thousands using… diff.
Regarding Africa, we’ve established 1 is a feature and 2 is a non issue, so I’d copy it again.
This approach scales both as the team scales and as the infrastructure scales. Teammates can read and comprehend much more easily than hierarchies of overrides, and changes are naturally scoped to pieces of the whole.
Sometimes things need to be complex -- well that's okay. The real trick is to not put complexity into places it doesn't belong.
I think Big Orgs need to develop younger promising talent by letting them build small green fields projects. Essentially fostering startups inside the organisation proper. Let them build and learn from mistakes (while providing the necessary knowledge; you can actually learn most of this from books, but experience is the ultimate teacher). Otherwise you end up with 5 year experienced people who cannot design themselves out of a paper bag.
Watch ‘Simple made Easy’ by Rich Hickey; a classic from our industry. The battle against complexity is ever ongoing. https://youtu.be/SxdOUGdseq4?feature=shared
* Our coding standards require that functions have a fairly low cyclomatic complexity. The goal is to ensure that we never have a a function which is really hard to understand.
* We also require a properly descriptive header comment for each function and one of the main emphases in our code reviews is to evaluate the legibility and sensibility of each function signature very carefully. My thinking is the comment sort of describes "developer's intent" whereas the naming of everything in the signature should give you a strong indication of what the function really does.
Now is this going to buy you good architecture for free, of course not.
But what it does seem to do is keep the cognitive load manageable, pretty much all of the time these rules are followed. Understanding a particular bit of the codebase means reading one simple function, and perhaps 1-2 that are related to it.
Granted we are building websites and web applications which are at most medium fancy, not solving NASA problems, but I can say from working with certain parts of the codebase before and after these standards, it's like night and day.
One "sin" this set of rules encourages is that when the logic is unavoidably complex, people are forced to write a function which calls several other functions that are not used anywhere else; it's basically do_thing_a(); do_thing_b(); do_thing_c();. I actually find this to be great because it's easy to notice and tells us what parts of the code are sufficiently complex or awkward as to merit more careful review. Plus, I don't really care that people will say "that's not the right purpose for functions," the reality is that with proper signatures it reads like an easy "cliffs notes" in fairly plain English of exactly what's about to happen, making the code even easier to understand.
Sometimes you do have the domain expertise to make the judgment call.
A recent example that comes to mind is a payment calculation. You can go ahead and tie that up in a nice reusable function from the get go - if you've ever dealt with a bug where payment calculations appeared different in some places and it somehow made it in front of a customer you're well aware of how painful this can be. For some things having a single source of truth outweighs any negatives associated with refactoring.
It explains so much of what has been bothering me about what I work on at work, and now I understand why and some of what to do about it.
Otherwise I mostly agree.
In order to get a sense of what code is harder to understand you will do better to read code and have others read your code. A good takeaway is to keep this in mind (amongst many other factors) and to understand code needs to be maintained, extended, adapted etc.
The ideas are still useful. The danger is blindly applying rules. As long as the reader knows not to apply any of the suggestions if they don't understand why and have relevant experience ;)
https://github.com/fzipp/gocyclo
> * We also require a properly descriptive header comment for each function and one of the main emphases in our code reviews is to evaluate the legibility and sensibility of each function signature very carefully. My thinking is the comment sort of describes "developer's intent" whereas the naming of everything in the signature should give you a strong indication of what the function really does.
https://github.com/mgechev/revive
> Now is this going to buy you good architecture for free, of course not.
It's not architecture to tell people to comment on their functions.
Also FTR, people confuse cyclomatic complexity for automagically making code confusing to the weirdest example I have ever had to deal with - a team had unilaterally decided that the 'else' keyword could never be used in code.
What made your team decide on that rule? Could your team decide to drop it since it hinders improving the design of your code?
Not weird at all:
https://medium.com/@matryer/line-of-sight-in-code-186dd7cdea...
Not everything is complicated, most functions don't need comments, why require it? Just fix complexity when it arises. Don't mandate that you can't make any complexity.
https://eslint.org/docs/latest/rules/no-else-return#rule-det...
But never using it is crazy.
Maybe one day we will abstract it away like the goto keyword (goto is a keyword in Go, and other languages still, but I have only seen it used in the wild once or twice in my 7 or 8 years of writing Go)
Goto is still used in almost every language, but it's abstracted away, hidden in loops, and conditionals (which Djikstra said was a perfectly acceptable use of goto), presumably to discourage its direct use to jump to arbitrary points in the code
Edit: sometimes the comments are the best of all evils, and you should use them to explain the constraints that led to this code - they just shouldn't be mandatory.
This is really a key takeaway here: Always keep your audience in mind. When programming, you have two audiences: the machine executing the code, and fellow programmers maintaining the code. Both are important, but the latter is often neglected and is what the article is about. Optimize for your human audience. What will make it easier for the next person to understand this? Do that.
Like public speaking or writing an article. A great talk or a article happen when the speaker/author knew exactly how the audience would perceive them.
(I sometimes "ask" questions for something it took me a few back and forths through code to get so they'd think about how it could be made clearer)
Unfortunately, most people focus on explaining their frame of mind (insecurity?) instead of thinking how can they be the best "teacher".
I feel this in my soul. But I'm starting to understand this and accept it. Acceptance seem to lessen my frustration on discussing with architects that seemingly always take the opposite stance to me. There is no right or wrong, just always different trade offs depending on what rule or constraint you are prioritizing in your mind.
Why do they insist on A over B? What trade offs were considered? Why are these trade offs less threatening than other trade offs? What previous failures or difficulties led them to put such weight on this problem over others?
Sometimes it's just ego or stubbornness or routine¹. That can and should be dismissed IMO. Even if through these misguided reasons they choose the "right" architecture, even if the outcome turns out good, that way of working is toxic and bad for any long term project.
More often, there are good, solid reasons behind choices, though. Backed with data or science even. Things I didn't know, or see different, or have data and scientific papers for that "prove" the exact opposite. But it doesn't matter that much, as long as we all understand what we are prioritizing, what the trade offs are and how we mitigate the risks of those trade offs, it's fine.
¹ The worst, IMO, is the "we've always done it like this" trench. An ego can be softened or taken off the team. But unwillingness to learn and change, instilled in team culture is an almost guaranteed recipe for disaster
Likewise if someone notices a bug in the method, you then have to go through and figure out which copies have the same bug, and fix each one, and QA test each one separately.
The proper approach is to make a judgement call based on how naturally generic the method is, and whether or not the existing use cases require custom behaviours of it (now or in the near future).
As an analogy, when writing a book, it's the difference of not repeating the opening plot of the story multiple times vs replacing every instance of the with a new symbol.
By way of analogy for why the two configs are different, for example Two beaches are not the same because they both have very similar sand.
You really have two different configs.. You also have one set of configs. You didn't set up an application that also fetches some config that is already provided. It would be like having a test flag in both config and database, sane flag - two places.
Where config duplication goes bad is when repeatedly the same change is made across all N, local variations have to be reconciled each time and it is N sets of testing you need to do. Something like that in code is potentially more complex, more obviously a duplication of a module, just more likely to be a problem overall.
Perhaps one day you will. I'm a dev who worked with infra people who had your philosophy: many copy pasted config files.
Sometimes I needed to add an env var to a service. Expressing "default to false and only set it to true in these three environments" took changing about 30 files. I always made mistakes (usually of omission), and the infra people only ever caught them at deployment time. It was hell.
With that said , get good at jumping into definitions, finding all implementations, then jumping back where you were. With a good IDE you can do that at the speed of thought (in IntelliJ that’s Cmb+b, Cmd+Alt+b, Cmd+[ on Mac). I only open more than one file at the same time when comparing them. Otherwise it’s much easier to jump around back and forth (you can even open another function inline if you just want to take a Quick Look, it’s Alt+Space). Don’t use the mouse to do that, things you do all the time can be made an order of magnitude faster via shortcuts. Too many developers I see struggle with that and are embarrassingly slow moving around the code base!
There's also the question of exhaustiveness checking. With visitors, you can typically opt-in to either checking that you handle everything. Or use the default no-ops for anything that you're not interested in.
So if you look at compilers for languages with pattern matching (e.g. Rust), you still see... visitors! E.g.: https://github.com/rust-lang/rust/blob/64a99db105f45ea330473...
That’s a stance of acceptance, however I’d say that there are people who are absolutely wrong by most metrics sometimes and also stubborn to the point that you’ll never convince them. Ergo, the frustration is inevitable when faced with them.
It's a delicate balance we need to keep in mind between many of:
- maintainable code
- getting things done
- feeling of accomplishment
- feedback loop speed
- coaching
- motivation and emotional state ("why are they pestering me, the code works, I just want to feel productive and useful: this was hard enough to get right as it is")
...and more!
At the same time, some do get the point, but getting readable code is really an art/craft in itself, and nothing but experience and learning to look at it from outside is the main driver to learning.
Architecture is another topic entirely and the scope is higher abstractions across multiple systems.
Also, people confuse familiar with simple, they tend to find things simple if they are familiar, even if they are complex (interwine a lot of different things).
This is why I want to get into system programming. At least less business logic. They are still going to be complicated, but I feel it's a lot nicer to, e.g. go from the bottom of the process struct than the bottom of some half-ass business stakeholder!
If two countries happen to calculate some tax in the same way at a particular time, I'm still going to keep those functions separate, because the rules are made by two different parliaments idependently of each other.
Referring to the same function would simply be an incorrect abstraction. It would suggest that one tax calculation should change whenever the other changes.
If, on the other hand, both countries were referring to a common international standard then I would use a shared function to mirror the reference/dependency that they decided to put into their respective laws.
This is true. However, very few people can clearly explain all the rules.
If they can, they have understood the system and are qualified.
Visitors in the linked example are real classic visitors. The code _within_ the visitor methods, of course, uses pattern matching, but the pattern itself is not materially different from C++.
Exhaustiveness checking for pattern matching is also "best effort" for complex matching.
And in general the primary benefit of such restriction is to reduce cognitive load. Scheme is easier than C++. The downside of such restriction is loss of expressiveness. Whether the net benefit is good depends on how these two things trade off. Experience and developer preference are inputs to that equation, which is why devs fight over coding guidelines. But I think it's helpful to boil it down in this way at a high level.
The ideal is smaller language where the expressiveness you've cut away is only rarely useful, and often error-prone.
One "rule" I try to meta-promote is — working code is the first step, and a great foundation to then proceed to clear and maintainable code.
Another, is that code reviews are first-class citizens deserving mindfulness.
Arbitrary structure rules like "do_thing_a(); do_thing_b(); do_thing_c();" also is not unless you can explain how this helps make it easier to understand compared to say, one big function with "// DO THING A" comments.