Estimates are difficult for developers and product owners

One thing that changed my way of thinking about estimates is reading that 86% of engineering projects, regardless of what kind of engineering (chemical, infrastructure, industrial, etc) go over budget (in time or money).

Missing estimates isn't unique to software, but it's common across all engineering fields.

Physical real-world projects include a buffer for this, called "contingencies" or "change orders" so that if a requirement changes or they discover something like previously unknown site geology that will require changes to the foundation they can absorb it. Based on a large history of similar projects their estimates are usually pretty good but occasionally they will run over.

What's wild about software is we don't do 1/10th as much planning as "real" engineering projects do, and also find ways to create problems akin to the ones they run into from interfacing with an unpredictable real-world environment, by inventing ways for all our platforms and libraries and such to be insanely unstable.

If you're not subject to the batshit insanity of the broader software market, and you crank the planning up a little closer to what "real" engineering does, software delivery gets extremely predictable. See: NASA's Space Shuttle software development processes.

(There are actual, not self-inflicted, problems with software development that "real" engineering doesn't see, though—for one thing, you can't on a whim completely restructure a building that's already been built, and for another, you generally don't have to design real-world engineering projects to defend against the case of intentional sabotage by someone who's just e.g. using the bathroom—it may happen, but it's rare and short of wacky Mission Impossible type plans, busting a water pipe in the bathroom isn't going to get you access to super-secret documents or anything like that)

> but occasionally they will run over

86% is more than "occasionally".

Even accounting for contingencies most of civil engineering projects go way over budget.

Two elements (the first quite obvious, the second not really) seem to be particularly common in overruns:

- the bigger the project the likelier the overrun. Small road projects tend to be over estimated, complex rail projects are virtually always way underestimated, mega projects are never close to the budget.

- the lengthier the planning and pre-construction phase the likelier the overrun. This is particularly interesting because it's counter intuitive: you would expect that the more analysis is done, the more accurate the estimates, but experience tells us the truth is the very opposite.