One of the hardest problems in software engineering is estimating the effort of a project. We often throw around the advice of “just double whatever you think it’ll be”. And sure, that might work, but it won’t ever make you better at estimating.

If all you’re doing is underestimating and then doubling up, you’re not getting better at the practice of estimating. So how can you get better at estimating?

Firstly, of course, getting good at estimating takes practice. You won’t get good at estimating software projects if you don’t do that. That means, initially, you need to have the space to estimate in a safe environment, where your estimates don’t result in hard deadlines for your team to meet.

A sustainable approach to estimating work

My approach to estimating work has developed over time, and I think where I’m at now is a good place to be, and it breaks down into three rough pieces.

1. Do a deep technical scope
2. Compare against similar work that’s been done in the past in the same (or a similar) codebase
3. Break large projects down into multiple smaller ones

Let’s dig into each of those.

Deep technical scoping

To make a good estimate, you need to take the time to plan it. With sufficient experience and domain knowledge, you might be able to give a rough estimate with very limited information, but diving deep into what your implementation will look like will give you much more certainty.

So what’s the right level of detail? This will depend on the specifics of the project, but a good guideline is to break it down to the point where every “chunk” looks similar to a piece of work you’ve done in the past, ensuring that a chunk is somewhere between half a day and two days, and absolutely no bigger than three days.

For example, good chunks may be:

1. Foundations: write down what your database table(s) and models will look like
2. CRUD operations: if your feature requires CRUD-style operations, you can usually fit that into a single chunk. Write down the API request and response payloads, and URL paths here, and maybe include a sketch of the frontend for it.
3. A more complex piece: complex endpoints that do a lot, or async jobs that have some complexity are usually good to separate.

Generally, you want to ensure that each chunk could be merged into your codebase separately (but not necessarily released in the product, e.g using feature flags to control the rollout).

The hardest bits to generalise technical scoping for are your complex bits. For those, it’ll be very dependent on the context of your application and project. Regardless, it’s often useful to dig into code specifics for more complex pieces.

Code-level detail doesn’t just give you as the person estimating this confidence. It also ensures that when it comes to implementing, the person building it can implement it based on a well-considered and reviewed plan, rather than coming up with a complex solution at implementation-time. This is often what slows projects down: being forced to come up with a solution to a complex problem on the fly. This is also often where things go wrong - when you’re implementing, you rarely have the space to consider edge cases and future concerns, and likely end up building something sub-optimal.

By solving the technical problems ahead of time, you’re spending a bit more time up front, to gain a lot more certainty when building.

A good technical chunk is usually between one and three paragraphs of text and/or diagrams (sometimes you may not need much text at all!)

Creating supporting documents for the most complex tasks

If you’re dealing with a really complex problem, you may want to create a completely separate document to compare the available options. This may still just be one technical chunk (i.e a couple days of work), but the implementation may have long-term impacts on the product experience and maintainability of your platform. For example, if you need to do something like handle delegated authentication (allowing your customers to create an authenticated page for their customer), you may want to spend a full two or even three days just looking at the options available to you and documenting those in a separate document, and then writing implementation details for the chosen solution in your technical scoping document.

Don’t skip the details

One of the easiest mistakes you can make is to progressively make your technical scope less and less detailed with each chunk. As time goes on, you start feeling pressure to complete scoping on time (maybe you allocated two days, and you’re halfway through day two, but only halfway through the technical scope).

Don’t fall into the trap of forcing yourself to complete scoping on time. By cutting out details from your scoping document, you may save half a day now, but you’re almost guaranteed to pay a tenfold penalty for this down the line, as you’re hit with 5 days of work you underestimated because you didn’t take the time to actually look at the codebase for one or two chunks.

Comparing against similar work

Once you’ve created your technical chunks, review them to see what is comparable to work you’ve done in the past. If you’re finding that a technical chunk is too big to be comparable, try to break it down further into something that can be compared. And if you have a chunk that is nothing like anything you’ve built before, write that down. It’s critical when estimating that you’re clear about which parts are uncertain. To continue with the previous example, you should be very certain about the time it takes to build a CRUD API, but it’s totally fine to say “I think it’s around 2-3 days to build magic link sign-in for delegated auth, but we’ve never done anything like it before, so this could extend significantly”.

You want most of your chunks to be between half a day and three engineering days, so if you see many of them stretching beyond that, you should probably break things down further.

On that note, you should be estimating in engineering days, i.e the number of days for a single engineer. You should also estimate how many engineers can effectively be put on the project, which will be based on how much parallel work can be done. Some projects have a lot of independent pieces of work and you may be able to put your entire team on it, while other projects can’t effectively be done by more than two people.

Create more projects

Once you’ve estimated every chunk, add it up, and figure out your total. At this point, if your numbers add up to more than 3 weeks of real time (roughly engineering days / # of engineers > 15), it’s probably too big.

Generally, I’d recommend keeping projects to 1-2 weeks, as maintaining momentum with your project team can be hard beyond that. If you’re shipping something big, break it down into multiple smaller projects that you can ship sequentially. It may seem like it won’t make a big difference, but giving your team a breather after every milestone can be a great reset. It can also be a good opportunity to rotate team members - you may just keep the project lead throughout the series of projects, while swapping out collaborators to give more team members the context.

Get your work reviewed

Once you’ve written your technical scope, it’s critical that you get it reviewed. Here, you want it at the very least to be reviewed by 1-2 engineers on your team. For more complex projects, particularly if you’ve had to write a separate supporting document for one of the chunks, you’ll likely want to get a senior engineer from another team to also review it, and get some feedback on your approach.

You don’t have to be 100% accurate

Remember that your technical scope is a guess at the end of the day. It’s how you think something should be built. Once you start building, you’ll almost certainly realise something doesn’t make sense - you realise an API endpoint needs more data, or that something needs to be an array rather than a single item. That’s fine. Since you scoped each chunk down to just a couple engineering days, unexpected changes should remain small, and allow you to adjust your project plans early and often.

Switch it around

Once you’ve done this a few times, you’ll build a muscle for writing scoping documents, at which point you can parallelise a lot of this. You’ll likely start noticing early on in scoping that “this seems like a lot of work for one project, let me figure out where I can split it”, allowing you to break it apart early, and then scope each project one by one.

Remember that the goal of technical scoping and estimating is to help you build product predictably, so that you can safely make commitments to customers when you need to, and build accurate roadmaps.