AI makes tech debt more expensive

> However, in ‘high-debt’ environments with subtle control flow, long-range dependencies, and unexpected patterns, they struggle to generate a useful response

I'd argue that a lot of this is not "tech debt" but just signs of maturity in a codebase. Real world business requirements don't often map cleanly onto any given pattern. Over time codebases develop these "scars", little patches of weirdness. It's often tempting for the younger, less experienced engineer to declare this as tech debt or cruft or whatever, and that a full re-write is needed. Only to re-learn the lessons those scars taught in the first place.

I call them warts, but yes agree, especially an industry that does a lot of changing, for example a heavily regulated one.

I recently watched a team speedrun this phenomenon in rather dramatic fashion. They released a ground-up rewrite of an existing service to much fanfare, talking about how much simpler it was than the old version. Only to spend the next year systematically restoring most of those pieces of complexity as whoever was on pager duty that week got to experience a high-pressure object lesson in why some design quirk of the original existed in the first place.

Fast forward to now and we're basically back to where we started. Only now they're working on code that was written in a different language, which I suppose is (to misappropriate a Royce quote) "worth something, but not much."

That said, this is also a great example of why I get so irritated with colleagues who believe it's possible for code to be "self-documenting" on anything larger than a micro-scale. That's what the original code tried to do, and it meant that its current maintainers were left without any frickin' clue why all those epicycles were in there. Sure, documentation can go stale, but even a slightly inaccurate accounting for the reason would have, at the very least, served as a clear reminder that a reason did indeed exist. Without that, there wasn't much to prevent them from falling into the perennially popular assumption that one's esteemed predecessors were idiots who had no clue what they were doing.

Louder for the people in the back. I've had this notion for quite a long time that "tech debt" is just another way to say "this code does things in ways I don't like". This is so well said, thank you!

Hahaha, Joel Spolsky predicted exactly that IN THE YEAR 2000:

https://www.joelonsoftware.com/2000/04/06/things-you-should-...

Imo real tech debt is when the separation between business logic and implementation details get blurry.

Rewrites tend to focus all in on implementation.

> Only to spend the next year systematically restoring most of those pieces of complexity as whoever was on pager duty that week got to experience a high-pressure object lesson in why some design quirk of the original existed in the first place.

Just to emphasize the point: even if it's not obvious why there is a line of code, it should at least be obvious that the line of code does something. It's important to find out what that something is and remember it for a refactor. At the very least, the knowledge could help you figure out a bug a day or two before you decide to pore over every line in the diff.

In my refactoring I always refer to that as Chesterton's Fence. Never remove something until you know why it was put in in the first place. Plenty of times it's because you were trying to support Python 3.8 or something else obsolete, and a whole lot of the time it's because you thought that the next project was going to be X so you tried to make that easy but X never got done so you have code to nowhere. Then feel free to refactor it, but a lot of the time it's because of good reasons that are NOT obsolete or overtaken by events, and when refactoring you need to be able to tell the difference.

https://www.chesterton.org/taking-a-fence-down/ has the full cite on the names.

Code that has thorough unit and integration tests, no matter how old and crusty, can be refactored with a good deal of confidence, and AI can help with that.

Times have changed. Code now does acquire bugs just by sitting there. Assholes you depend on are changing language definitions, compiler behavior, and libraries in a massive effort concentrated on breaking your code. :)

There is a difference between "this code does things in ways I don't like" and "this code does things in ways nobody likes"

100% hear this and I know as a developer at a big company I have no say over the business side of things but there's probably something to be said for we should all push for clear logical business processes that make sense. Take something like a complicated offering of subscriptions, it's bad for the customer, it's bad for sales people, it's bad for customer support, honestly it's probably even bad for marketing. Keep things simple. But I suppose those complexities ultimately probably allow for greater revenue as it would allow for greater extraction of dollars per customer e.g. people who met this criteria are willing to pay more so we'll have this niche plan but like I outlined above at what cost? Are you even coming out ahead in the long run?

Incidentally the person who really convinced me to stop trying to future-proof made a point along those lines. Not in the same language, but he basically pointed out that, in practice, future-proofing is usually just an extremely efficient way to litter your code with Chesterton's Fences.

It acquires bugs, security flaws, and obsolescence from the operating system itself.

Golang really is the best when it comes to backwards compatibility. I'm able to import dependencies from 14 years ago and have them work with 0 changes

I got really 'lucky' in that the first major project I ever worked on was future-proofed to high heaven, and I became the one to maintain that thing for a few years as none of the expected needs for multiple layers of future-proofing abstraction came to pass. Oh but if we ever wanted to switch from Oracle to Sybase, it would have been 30% easier with our database connection factory!

I never let that happen again.

> Assholes you depend on are changing language definitions, compiler behavior, and libraries in a massive effort concentrated on breaking your code. :)

Big Open Source is plotting against the working class developer.

This can be a huge road block. Even if the developer who wrote the code is still around, there's no telling if they will even remember writing that line, or why they did so. But in most projects, that original developer is going to be long gone.

I leave myself notes when I do bug fixes for this exact reason.

> Over time codebases develop these "scars", little patches of weirdness. It's often tempting for the younger, less experienced engineer to declare this as tech debt or cruft or whatever, and that a full re-write is needed. Only to re-learn the lessons those scars taught in the first place.

Do you have an opinion when this maturity is too mature?

Let's say, you would need to add a major feature that would drastically change the existing code base. On top of that, by changing the language, this major feature would be effortless to add.

When it is worth to fight with scars or just rewrite?

There is a pretty well known essay by Joel Spolsky (which is now 24 years old!) titled "Things You Should Never Do" where he talks about the error of doing a rewrite: https://www.joelonsoftware.com/2000/04/06/things-you-should-... . While I don't necessarily agree with all of his positions here, and given the way most software is architected and deployed these days some of this advice is just obsolete (e.g. relatively little software is complete, client-side binaries where his advice is more relevant), I think he makes some fantastic points. This part is particularly aligned with what you are saying:

> Back to that two page function. Yes, I know, it’s just a simple function to display a window, but it has grown little hairs and stuff on it and nobody knows why. Well, I’ll tell you why: those are bug fixes. One of them fixes that bug that Nancy had when she tried to install the thing on a computer that didn’t have Internet Explorer. Another one fixes that bug that occurs in low memory conditions. Another one fixes that bug that occurred when the file is on a floppy disk and the user yanks out the disk in the middle. That LoadLibrary call is ugly but it makes the code work on old versions of Windows 95.

> Each of these bugs took weeks of real-world usage before they were found. The programmer might have spent a couple of days reproducing the bug in the lab and fixing it. If it’s like a lot of bugs, the fix might be one line of code, or it might even be a couple of characters, but a lot of work and time went into those two characters.

> When you throw away code and start from scratch, you are throwing away all that knowledge. All those collected bug fixes. Years of programming work.

Put another way, sometimes code is complex because it has to be.

> Sure, documentation can go stale, but even a slightly inaccurate accounting for the reason would have, at the very least, served as a clear reminder that a reason did indeed exist.

Which is borderline the reason for version control: Do a git/svn blame on that line, find what commit it was added, and see what the commit message was. Bonus points if it links to a case on a system you still use. Sure the commit message can be useless, but it's at least something you're forced to enter when committing code, rather than external documentation that can be missed and now be misleading. Version control can even show you that codebase at time that change was made so you can see it in context (which has saved me a few times, showing what something was added for so I could confirm a suspicion).

In general, when people say this sort of thing, if I dig into what exactly they're doing I discover they're importing half of npm/pypi/etc.

My code doesn't acquire bugs by sitting there in 2024 any more than it did in 2004. On most projects these days I'm using Django + Preact + HTM. Preact and HTM get loaded from static files by my root Django template. My PyPi dependencies are pinned to specific versions, and usually I have <10 (usually it's just Django and Django REST framework, sometimes it's even just Django).

> ...if we ever wanted to switch from Oracle to Sybase...

Yeah like Oracle would ever let that happen

> *"Things You Should Never Do" where he talks about the error of doing a rewrite

As a funny aside, I actually noticed this in a completely different field, serial stories on the web (mostly on RoyalRoad)!

Occasionally an author will attempt a rewrite of a story either because the feedback was very critical, or they did not like where their own story had ended up.

I have yet to see a single example for a truly successful rewrite, where the rewrite was really significantly (or at all) better than the original. Usually the rewrite will not get any better ratings or more readers than the first draft - and for good reasons.

There will be improvements, but it will be on the edges. At the core it still remains the same story with the same problems, and some style changes or some improved dialogs don't change that.

----

By the way, there is an old 2016 HN thread with 106 comments "When to Rewrite from Scratch – Autopsy of Failed Software" -- https://news.ycombinator.com/item?id=11553813

----

A rewrite story I heard a long time ago and that I think would actually work best when the issues are severe was from a company that lost all their code (I don't remember the context, it was not data loss). They had spent many years to get to where they had been when they lost everything. They thought it would take almost as many years to get there again, but they started anyway. Turned out they were done in only half a year this time, and much better!

I think having to work with and around your old code (or story, in the RoyalRoad example) is a severe limit on how much you can improve. Your thoughts are not free, most of your mental effort will be around reusing the old code.

That is my own experience too: Writing the software is not my bottleneck. It's finding out what to write in the first place, and the many many small agonizing decisions along the way. I now see that meta knowledge is far more important. For very large projects it may be more difficult though.

I did this myself once, in the early Internet growth days. The company had its own equivalent of PHP (which was still pretty new at the time) and a business software based on it. I was tasked with refactoring the 1.0 version. I threw the code away after a brief look and rewrote from scratch. I did it because I believed having to consider the existing code would be much slower than writing new.

I have no complaints about the 1.0 version, the first version is always limited by the by then still low comprehension of the problem. I think version 2.0 releases might benefit the most from just throwing the 1.x code away and starting fresh, if the understanding of the problem evolv3ed substantially during - and through - the development.

Or phrased positively, Big Open Source keeping the working class developer employed

IME that kind of thing is more likely to make it 300% harder.

This idea of easy, worry-free database replatforming strikes me as kind of a shibboleth for identifying people who’ve never done it before. In reality they all have subtle differences in semantics and query optimization behavior that mean that every touch point needs close attention to make sure you understand how the behavior in that part of the system changes (assume it will change) and if that change is acceptable. Thinking abstraction layers can eliminate the need for close attention to a DBMS port is the software engineering equivalent of thinking adaptive cruise control means you can play Slay the Spire while driving to the office.

It was even more ludicrous in this case because a ton of the app logic was inside stored procedures.