I am often asked this question during SLIM Training classes.  I remember wondering about that myself.  It is a logical question since SLIM-Estimate workbooks are often imported into SLIM-Control to create the baseline project plan.  The answer is ‐‐ they are not directly related, because uncertainty ranges, probability curves, and control bounds are designed to perform different tasks.  This post is the first in a series looking at risk associated with an estimate, risk of your project plan, and handling deviations from the plan.

What are we talking about?

The first thing we need to do is define some very important terms that are often misused (I am the first to admit I have been guilty!).  I went to good old Dictionary.com and looked up the following:

QSM is pleased to welcome Andy Berner to our development team as a Senior Software Engineer. He will be supporting the product development team on new SLIM software estimation, forecasting, and benchmarking products as well as the IBM Rational integrations. As an IBM Rational Partner, QSM has had the privilege of working with Andy over the last several years designing and implementing integrations between the SLIM Tool Suite and the IBM Rational applications Rational Team Concert, Rational Focal Point, and Rational Method Composer.  

Andy Berner joins our team with experience in a wide variety of areas in software development. Andy came from IBM where he was lead architect for enablement and strategy in the Ready for IBM Rational program. Andy has done extensive consulting on software development methods and tools, recently focusing on integrations of tools and team members throughout the software lifecycle. Prior to IBM, Andy spent 11 years at EDS. In a former life, Andy was a research mathematician and teacher, and is now looking forward to helping QSM customers improve their ability to manage and control their projects.

A few weeks ago, Thomas C. Redman posted Demand the (Right) Right Data on the Harvard Business Review blog, about how managers should set the bar higher, in terms of data.

Why are managers so tolerant of poor quality data? One important reason, it seems to me, is that most managers simply don't know that they can expect better!  They've dealt with bad data their entire careers and come to accept that checking and rechecking the "facts," fixing errors, and accommodating the uncertainties that using data one doesn't fully trust are the manager's lot in life.

Although Redman suggests that managers should demand higher quality data, I immediately thought about how to check the quality of SLIM-DataManager databases using the Validate function and SLIM-Metrics.

If you're using SLIM-DataManager to create your own historical database, you can use the Validation feature to help you demand the (right) right data.  The Validation feature in SLIM-DataManager analyzes the projects in your database, highlights suspect projects, and offers a brief explanation tool tip.  Simply go to File|Maintenance|Validate to run this feature and wait for SLIM-DataManager to analyze your database.  If SLIM-DataManager detects anomalies, it will highlight that project in blue.  If you hover over that project, a tooltip will explain what is wrong with that project data and what you need to take a second look at.

A colleague of mine recently sent me a blog post explaining the difference between project contingency and padding.  The blogger made the distinction that padding is what often gets added to an individual’s estimate of the effort required to perform a task (in her example, a software development task) to account for project ‘unknowns’.  The estimator determines the most likely required effort, then pads it with a little more effort in order to arrive at an estimate to which he or she can commit.  Thus, padding represents an undisclosed effort reserve (and implied schedule reserve) to buffer against potential risk.  Contingency reserve, she explains, is “an amount of money in the budget or time in the schedule seen and approved by management.  It is documented.  It is measured and therefore managed.”  Ms. Brockmeier is correct in promoting contingency as the better management tool.  The challenge is having a method to measure and document this contingency and the known unknowns it is buffering.

Implicit Risk Buffer

Padding is a natural result of bottoms-up, effort-based estimation techniques.  Estimating low-level WBS elements creates more opportunity for padding, because the number of unknowns grows with the task list.  The estimator is consciously or unconsciously assessing the risk of each task, considering its dependencies and complexities.  What is being implied in the effort estimate is: 1] an assessment of product size and complexity, and 2] a productivity valuation.

If you were unable to attend our most recent webinar, Successful Estimating Processes Using the SLIM API, a replay is now available.

How do best in class development organizations achieve maximum return on investment from their estimation programs? By leveraging the SLIM API for integrations between estimation tools and detail-oriented products, development teams are able to simplify estimation processes and broaden the estimation program user base. Presented by Carl Engel of IBM Global Services, Scott Lancaster of State Street, and Larry Putnam, Jr. of QSM, this webinar explores two successful implementations of the SLIM API between third party tools and the SLIM Suite. 

Carl Engel is the Estimating Program Manager for IBM's Global Business Services responsible for the development and deployment of performance benchmarking and estimating process, methods and tools including the support for nearly 1,000 SLIM Suite users. Carl has been with IBM for 12 years as an Associate Partner and has had previous roles as the program manager for IBM's project management methodology and tools. He is an IBM certified Executive Project Manager, PMP with over 30 years of program and project management experience primarily in very large scale efforts in the nuclear industry and U.S. National Laboratories.

Anyone who has gambled (and lost) knows the sting of losing.  In 1979, Daniel Kahneman and Amos Tversky, pioneers in the field of behavioral economics, theorized that losses loom larger than gains; essentially, a person who loses $100 loses more satisfaction that what is gained by someone who wins $100. Behavioral economics weaves psychology and economics together to map the irrational man, the foil of economics' rational man. 

How can I leverage this theory for software development?

According to the QSM IT Software Almanac (2006), worst in class projects took 5.6 times as long to complete and used roughly 15 times as much effort with a median team size of 17, and were less likely to track defects. 

One way you can leverage your worst in class projects would be to use them as history files in SLIM-Estimate, which would adjust PI, defect tuning, etc., to match how you have developed software in the past. Don Beckett recently discussed how to tune effort for best in class analysis and design.

Another way to leverage your worst in class projects would be to build a "project graveyard," that is, a database of your organization's worst projects, and load it into SLIM-Metrics. In SLIM-Metrics, you can analyze duration, peak staff, average staff, and defects to view your own organization's weaknesses. Depending on how well documented your SLIM-DataManager database is, you could analyze some of the custom metrics that ship with SLIM-Metrics, such as reviewing who the project was built for (customer metric) and complexity.

On April 12, 2012 at 1:00 PM EDT, QSM will host a webinar focused on two successful implementations of the SLIM API presented by IBM's Carl Engel, State Street's Scott Lancaster, and QSM's Larry Putnam, Jr. 

How do best in class development organizations achieve maximum return on investment from their estimation programs? By leveraging the SLIM API for integrations between estimation tools and detail-oriented products, development teams are able to simplify estimation processes and broaden the estimation program user base. Presented by Carl Engel of IBM Global Services, Scott Lancaster of State Street, and Larry Putnam, Jr. of QSM, this webinar explores two successful implementations of the SLIM API between third party tools and the SLIM Suite. 

Carl Engel is the Estimating Program Manager for IBM's Global Business Services responsible for the development and deployment of performance benchmarking and estimating process, methods and tools including the support for nearly 1,000 SLIM Suite users. Carl has been with IBM for 12 years as an Associate Partner and has had previous roles as the program manager for IBM's project management methodology and tools. He is an IBM certified Executive Project Manager, PMP with over 30 years of program and project management experience primarily in very large scale efforts in the nuclear industry and U.S. National Laboratories.

The February issue of the DACS Journal of Software Technology focuses on Software Cost Estimation and Systems Acquisition. My contribution, which you can read here, addresses the challenges faced by estimators and the value of establishing a historical baseline to support smarter planning, counter unrealistic expectations, and maximize productivity.

Using several recent studies, my paper addresses the following questions:

• What is estimation accuracy, and how important is it really?
• What is the connection between the Financial Crisis of 2008 and software estimation?
• Why do small team projects outperform large team projects?
• How can you find the optimal team size for your project?

Read the full article.

In Part 1 of How Much Estimation? we noted that there is an optimal amount of time and effort that should be spent in producing an estimate based on the target cost of a project and business practice being supported.

In Part 2: Estimate the Estimate, we saw that the formula to calculate this optimal time (as measured at NASA)  calculates the Cost of Estimate as the Target_Cost raised to the power 0.35 (approximately the cube root of the Target Cost).  The factor that defines the business practice (either by early lifecycle phase or perhaps by the “expected precision” of the estimate) is a linear factor ranging from a value of 24 to a value of 115.

Those Caveats!

I mentioned that there were caveats with the calculation.  Here they are:

In Part 1 of How Much Estimation, we observed that both too much time and effort and too little time and effort spent on estimating are less than optimal.  Combining:

• The cost of producing an estimate—which is a function of the number of people working on the estimate and how long they work
• The cost of variance in the results of the estimate—that is, how much the estimate varies from experienced actuals and what that variance will likely cost the project.  This is typically a function of the number of unknowns at the time of estimating for which the project cannot easily adjust and which will require additional unplanned resources of time, effort, and staff.

We get a U-shaped curve, at the bottom of which is the optimal time: we’ve spent enough time and effort to minimize the sum of the cost of estimate and the cost of variance.

The question is: how to calculate this point?  It will not be the same for a very large complex project and a very small simple project.  Also we don’t want a complicated and time-consuming approach to calculate the cost of estimate—it should be quick and simple.

NASA’s Deep Space Network (DSN) projectⁱ developed a mechanism for this calculation based on two simple parameters:

Target Cost of Project

This is goal cost of the project as first envisaged in the project concept.  It is NOT the estimated cost of the project (which hasn’t been calculated yet).  Projects for which we expect and plan to spend a lot of money should clearly have more time and effort spent in estimating simply because more is at risk.