The notion of False Time Estimation Syndrome

Missing a deadline, or pulling an all-nighter the night before the delivery date are not rare occasions in the software development world.

Before Agile, the main issue was a strict planing happening so long before the development, that it left no room for any changes. But since Agile methodologies came to our lives all of that became a relic of the past.

Well... apparently not!

Most of us still spend a lot of time on planning, with countless meetings that should ensure everything goes as scheduled. But, we still struggle to be on time and find ourselves constantly under pressure.

This state is very unhealthy, it builds frustration and lowers the trust between the developers and the management.

As time goes by, it may drive people to leave the company and those who do stay are focused mainly on lowering any chance of overworking.

I found myself in similar situations over and over again. So, I started wondering what is the cause for it. I started paying attention to what I was doing wrong and were the process have gone bad. I researched different methodologies, searching for the essence in the promise they are trying to bring.

Although probably none of the things in this article will be new to you, it shows a different perspective arguing that behind most of the reasons for being constantly late lies one core cause. Rarely it is acknowledged to be a real problem, laying blame on other things and forcing you to apply changes to the way you work without any real improvement.

What it's all about?

So, what is wrong? Were we misled by the promise of Agile?

The answer is: No...

There might be a lot of reasons for being constantly late: entering a new domain, incorporating new technology or an inexperienced team.

But really, the one issue that stands behind them all is: False Time Estimation Syndrome.

It doesn't matter if you're using Scrum, Kanban or XP, in real life your planning relies on proper time estimation.

You can't plan a sprint without knowing how much time each task will take, and you can't match the work in progress to the capacity of the team in Kanban without correct time estimations.

Although Agile methodologies try to remove the hard deadlines and focus on velocity and continuous delivery, in reality you always have hard commitments to your customers, investors or colleagues. And it doesn't matter how you run your development as long as you deliver on time.

Why do we lie when providing time estimation?

Let's be honest, we all lie when providing time estimations. But the reasons for that aren't due to bad will or some sadistic desire to sabotage the project.

In most cases, we don't even know that we are falsifying our estimations.

Still, why do we do it?

Lack of experience

Lack of experience can be due to the simple fact that the developer, or the manager, are young and inexperienced. But that, in most cases, is not the issue. Such person is usually supervised more tightly and his estimations are examined more thoroughly.

More often, the so called "lack of experience" happens when your plan involves a domain that you're not an expert at. Usually, entering a new domain bares with it a lot of unknowns, you might think that you can finish the task at hand during some period of time, based on your past experience with similar tasks. But, once you start coding the unknowns become blockers and in extreme cases can send you back to the drawing board, eventually causing you to forfeit the deadline.

Pressure

This is a very common reason for providing false time estimations. It happens when your time estimations aren't aligned with the commitment of the product owner, the management or your peers. In this case you'll be pressured to deliver the same content in a non-realistic time frame. This will always force you to change your planning to accommodate someone else's timeline, neglecting any pitfalls you might have. Even worse, in order to save time, you might compromise on improper engineering, bad design and lower quality.

Passion

Deep down we are all hackers. We are full of passion to code and eager to create new and innovative things. Most of the time those are exactly the projects that bare unknowns in them.

We fear that if we won't provide an estimation that will be satisfactory by the management the project will be either taken from us or postponed until further notice. This could trigger us to give false time estimations in order to get the project we so much desire. But, yet again, this will make us accommodate someone else's will.

How to detect that you have the False Time Estimation Syndrome?

That is not as hard as it seems. All you need to do is pay attention to the situation when the planning is not aligned with the execution.

Any Agile methodology contains all the tools you need to do exactly that. The only thing left for you is to know their value and use them.

For example, in order to spot that the estimations were wrong a simple look at the Kanban board will show you how many tasks are stuck for a long time in the same state. If there are to many, it could point to a bottleneck caused by wrong estimations.

If you're using Scrum, not meeting the spring goals, sprint over sprint, can be a good indicator of this problem. Although, it is fine to transfer an unfinished task from one sprint to another, when you constantly unable to finish all planned tasks it usually indicates that you have a serious problem with your estimations.

But, that is not enough. You should also be minded to the development effort. Ask yourself, was there an unreasonable pressure on the developers? Did people put some crazy overtime just to make it on time? Did you cut corners along the way? Does this happen over and over again?

You might officially deliver on time, but under the surface wrong time estimations led your execution to be unbalanced wearing down your team as time passes.

Any version of retrospective will provide answers to those questions and will reveal any hidden frustration that your team might have due to bad planning.

The "magical" solution

I wish there was a magical solution, but unfortunately there isn't one.

It's a matter of maturity for everybody in the organization.

However, in order to achieve such maturity there are several guidelines you should follow:

Stop lying to yourself. The first thing you need to do in order to overcome the False Estimation Syndrome is to stop lying to yourself. Yes, to yourself! As I said earlier we seldom consciously lie to our peers. But, we constantly lie to ourselves. When you are estimating the task at hand, you should stop and check if you haven't missed anything. Pay attention to being too overconfident, even if you're 100% sure in your estimations things might still change along the way.

Bury your ego. Leave your ego out of the way. People that are driven by ego are blocked to suggestions and criticism from others. They blame everyone else except themselves, and afraid to receive feedback. If governed by ego, no methodology, no matter how perfect it is, will ever help you identify, and acknowledge, that your time estimations are wrong.

Never cut down on buffers. This is the most basic thing we all learn about planning: always leave buffers! But, this is also the first thing that is neglected. Although it is the most important tool to handle unexpected delays we often tend to squeeze more content into it. Mainly because it might seem, even if subconsciously, a waste of time.

Plan with stretch in mind. This point is tightly coupled to the previous one. In order to make the buffers safe from being cut down you should always have backup tasks that you can fill the buffer with, just in case you won't use it. But, it's important to communicate those tasks as stretch, otherwise it'll be expected from you to deliver them as well leaving you with the same problem.

Don't compromise on quality. Quality is always next in line to be cut down. Although it's no less important part of the development, there is always a tendency to drop unit tests and neglect performance testing. In the long run, this will make the situation even worse when the flood of production bugs will come crushing down on you, putting even more pressure on you and your team. There are a lot of ways to ensure quality, some of them don't require any extra effort, and can even leverage production to test your code. You should choose the one way that will insure quality and match your operational restrictions.

Reduce the delivery content. In many cases the requested content is full with unnecessary things. You must strive to identify and remove them. This is not an easy task, and usually involves deeper understanding of the product as well as the business side. But, this is the one thing that can let you earn the time you need, without compromising on proper engineering. Don't just whine about it, come up with several suggestions and a feasible plan for cutting out unnecessary content.

Stand up for what you believe. This might come as a shocker to you, but standing up for what you think is right is a major part of the maturity I was talking about earlier. Now, I'm not telling you to burn down bridges and pick random fights with your peers. But, don't back down just due to some notion of organizational hierarchy. You might not win all the arguments, but you'll make people think about your suggestions. If you just do what you're told to, you'll end up loosing control over your planing.

Constantly improve. We're all humans and make mistakes, and this is fine as long as we improve. It is a fundamental aspect of any Agile methodology. If you miss the deadline or have a lot of blockers, stop and revise what is wrong. A continues improvement, is the most important thing you need in order to overcome the False Estimation Syndrome.

Conclusion

The False Time Estimation Syndrome, might affect your productivity and ability to scale up your work.

Although it is rather easy to identify, it is the last thing to be recognized as a problem and treated as such.

It takes an effort to fix, and involves the participation of all parties in the organization.

But, doing so will allow your organization to enable growth, instead for constantly chasing after its tail and putting out fires.