Wednesday, August 6, 2014

You are bad at estimating

For the last 4 years or so at Korrelate we have been using Scrum. We had 2 week sprints, estimated stories and planned based on those estimates. In that time we have learned one very important lesson: we are very bad at estimating. In retrospect, this shouldn't have come as a surprise to anyone. There is a mountain of research across multiple fields that demonstrates just how bad expert estimates are. They are so bad that on average they are worse than randomly assigned numbers. If you think you are somehow different and you can do it better, you are wrong. You may be thinking that your estimates have been fairly accurate and your sprints largely successful. There are some reasonable explanations for this phenomena:

  1. You are doing a lot of very similar tasks. Given a history of nearly identical work it is possible to come up with fairly good estimates
  2. You have a large enough team that your bad estimates have roughly balanced each other out so far
  3. You are working a lot of extra hours
  4. You have yet to experience regression to the mean

Given that we know we are bad at estimating, what should we do? I propose a fairly simple change: weight all stories equally. That's right, don't estimate anything. This may sound crazy at first, but the evidence shows that equal weighting is on average better than expert estimates and as good or nearly as good as sophisticated algorithms. You would probably do better than your current estimates by basing them on the number of words in the story.

Now, obviously some stories will require more effort than others, we just don't have a good idea of which ones those are. I propose another change to help here: break every story up into the smallest pieces that make logical sense. Some stories will becomes epics that contain multiple smaller stories. If the story can't be sensibly broken up then just leave it. Do not make any attempt to equalize them either within the epic or against other stories, that's just a back door to estimating. I think this is the best attempt we can and should make to reduce the difference in effort between stories.

So, without estimates, how do you plan? This brings up the issue of sprints. My final proposal is that we abandon them as well. If you want to know how many stories the team is likely to complete over the next month, just take the average number of stories they have completed over the last few months. The time they took to finish is irrelevant and you can still follow trends in velocity over time and use them to provide better estimates about things like how adding another engineer will affect velocity and how long it will take to ramp them up. Resist the urge to add your "professional intuition" into the equation, you will only screw it up. Trust the data, not your gut.

I'd love to hear your personal experience with estimating or not estimating. Sprints vs continuous deployment or anything else related to improving the development process.

No comments:

Post a Comment