Estimates are difficult for developers and product owners

Hours is insane. But ultimately time is money and opportunity cost. Software engineering can’t be the only engineering where you ask the engineers how much something will cost or how much time it will take and the answer is “it’s impossible to know”. Even very inaccurate estimates can be helpful for decision making if they are on the right order of magnitude

I think software is one of those VERY rare things, where inaccurate estimates can actually be inaccurate by "orders of magnitude". After 20 years in the field, I still managed to use 2 months of time on a task that I estimated as 10 days.

The next natural progression of this line of discussion between "the business" and engineering is for them to agree on a time range as an estimate. Engineering doesn't want to say it'll be done in 6 weeks, but they feel okay saying it will take between 4 and 20 weeks so this estimate is accepted.

You can guess what happens next, which is that around week 8 the business is getting pretty angry that their 4-week project is taking twice as much time as they thought, while the engineering team has encountered some really nasty surprises and is worried they'll have to push to 24 weeks.

There's two things here that get overlooked.

First, people asking for estimates know they aren't going to get everything they want, and they are trying to prioritize which features to put on a roadmap based on the effort-to-business-value ratio. High impact with low effort wins over high impact high effort almost every time.

Second, there's a long tail of things that have to be coordinated in meat space as soon as possible after the software launches, but can take weeks or months to coordinate. Therefore, they need a reasonable date to pick- think ad spend, customer training, internal training, compliance paperwork etc.

"It is impossible to know" is only ever acceptable in pure science, and that is only for the outcome of the hypothesis, not the procedure of conducting the experiment.

A rule that has suited me well is to take an estimate, double it, and increase by an order of magnitude for inexperienced developers. So a task the say would take two weeks ends up being 4 months. For experienced developers, halve the estimate and increase by an order of magnitude. So your 10 days estimate would be 5 weeks.

10 days was already after I used this algorithm. Previous several tasks on that codebase were estimated pretty good. Problem with this is that some tasks can indeed take SEVERAL orders of magnitude more time that you thought.

One of the hardest problems with estimating for me is that I mostly do really new tasks that either no one wants to do because they are arduous, or no one knows how to do yet. Then I go and do them anyway. Sometimes on time, mostly not. But everybody working with me already knows, that it may be long, but I will achieve the result. And in rare instances other developers ask me how did I managed to find the bug so fast. This time I was doing something I have never before done in my life and I missed some code dependencies that needed changing when I was revieving how to do that task.

Something often overlooked in cost/schedule estimates is the nature of joint probability of actions slipping. Action A slips and causes action B to slip. I think software is tougher to estimate because the number of interfaces is often much higher, and sometimes more hidden, than in hardware.

The biggest estimation effort I ever took part in was a whole-system rewrite[1] where we had a very detailed functional test plan describing everything the system should do. The other lead and I went down the list, estimated how long everything would take to re-implement, and came up with an estimate of 2 people, 9 months.

We knew that couldn't possibly be right, so we doubled the estimate and tripled the team, and ended up with 6 people for 18 months - which ended up being almost exactly right.

[1]: We were moving from a dead/dying framework and language onto a modern language and well-supported platform; I think we started out with about 1MLOC on the old system and ended up with about 700K on the rewrite.

That's a factor four or five ir so, so still less than an order of magnitude.

as opposed to say building a house where framing can totally slip while we run electricity and build a roof floating in mid-air

software is only tougher to estimate if incompetent people (vast majority of the industry, like 4+ million) is doing the estimating :)

I’ll send my friend that has a construction company to build your next 3500 sq ft house for $13.6 million dollars :)

My take: if they have not done the work to get to at least some degree of a spec, and they won't give you time to review and investigate, they get nothing more than a vague, relative t-shirt size.

it is still better to give a range though because 1. it explicitly states the degree of unknown and 2. no boss is going to accept 4-20 weeks, which means you start talking about how you can estimate with better accuracy and the work required to do so, which is a major goal of planning & estimation.

> "as soon as possible after the software launches"

This isn't true, just desired, and is one of the main roots of the conflict here. OF COURSE you would like to start selling in advance and then have billing start with customers the instant the "last" pr is merged. That isn't a realistic view of the software world though and pretending it is while everyone knows otherwise starts to feel like bad faith. Making software that works, then having time to deploy it, make changes from early feedback, and fix bugs is important. THEN all the other business functions should start the cant-take-back parts of their work that need to coordinate with the rest of the world. Trying to squeeze some extra days from the schedule is a business bet you can make but it would be nice if the people taking this risk were the ones who had to crunch or stay up all night or answer the page.

Trying to force complicated and creative work into a fake box just so you can make a gantt chart slightly narrower only works on people a couple times before they start to resent it. 10x that if management punishes someone when that fantasy gantt chart isn't accurate and 100x that if the one punished is the person who said "it's impossible to know" and then was forced into pretending to know instead of the person doing the forcing.

My home construction slipped 6 months on 2 year build time. It happens in construction very often.

> software is only tougher to estimate if incompetent people (vast majority of the industry, like 4+ million) is doing the estimating :)

No, it is tough to estimate, but not only for incompetent people. And "incompetent" can be stretched to "don't know what he's doing", which is how I operate most of the time. I don't know what really needs to be done until it's done. Main part of my work is research on what actually needs to be done, then "just" implementing it. If I waited with estimating until I know what needs to be done, I would spend 3/4 time estimating and then 1/4 with clear understanding and good schedules (example description: I will be clicking keys for 5 hours).

> My home construction slipped 6 months on 2 year build time. It happens in construction very often.

Tangent, but I have at least 3 friends that would've (in retrospect) been nothing short of ecstatic if their home construction had "only" slipped 6 months on a 2 year timeline.

> Software engineering can’t be the only engineering where you ask the engineers how much something will cost or how much time it will take and the answer is “it’s impossible to know”.

Because it's not engineering at all. But even if it was, plenty of engineering projects are impossible to estimate - the ones that are doing something novel - and disliking that fact doesn't make it go away.

> Even very inaccurate estimates can be helpful for decision making if they are on the right order of magnitude

If what the business wants is an order-of-magnitude, they should ask for that; often (not always!) that's a lot easier.

That’s a bit of a strawman considering I was deliberate in saying hardware interfaces are limited and not saying they are zero. The number of interfaces in software is often going to be orders of magnitude greater. The network effects and failure modes will often increase geometrically with the number of interfaces. In fact, big construction design firms have tools to easily identify and mitigate the “clashes” you bring up and those tools tend to work well because there is a finite number and the designs are well-documented (as opposed to software where changes are relatively cheap and easy so they often occur without documentation)

Saying incompetence is the reason is a trivial rebuttal that ignores the central claim about complexity. It’s like saying “the reason why we don’t have a theory of everything is because we don’t have competent physicists”