Practical Software Measurement

Agile On-Time, But Is It Reliable?

With agile projects, we hear a lot about the planning benefits of having a fixed number of people with a fixed number of sprints.  All great stuff when it comes to finishing on time and within budget. But one of the things we also need to focus on is the quality of the software.  We often hear stories about functionality getting put on hold because of reliability goals not being met.

There are some agile estimation models available to help with this, and they can provide this information at the release level, before the project starts or during those early sprints. They provide this information by leveraging historical data along with time-tested forecasting models that are built to support agile projects. 

In the first view, you can see the estimate for the number of defects remaining. This is a big picture view of the overall release. Product managers and anyone concerned with client satisfaction can use these models to predict when the software will be reliable enough for delivery to the customer.

MTTD over Time

In the second view, you can see the total MTTD (Mean Time to Defect) and the MTTD by severity level. The MTTD is the amount of time that elapses between discovered defects. Each chart shows the months progressing on the horizontal axis and the MTTD (in days) improve over time on the vertical axis. 

Mean Time to Defect

These models are also valuable because they give us a good view of the risk associated with the software reliability. In the view below, you can see that there is a 99% chance that the MTTD will be less than half a day. This means that the software will only run part of a day without being impacted by a defect. If these are critical defects than we have some serious problems on our hands. So this is not a good time to deliver even though the fixed number of sprints is set to be completed - red flag to a product manager.

Agile Project Reliability Risk Profile

Defect estimation is extremely important with agile and any other type of software development method. We need to be able to see the release level defect estimates before we agree to a delivery date and budget. What good is it to deliver on time and within budget if the software isn’t going to work to a high standard of customer satisfaction?

Blog Post Categories 
Agile Quality Estimation Software Reliability