Outsourcing was supposed to make government IT executives’ lives easier. Yet in too many cases, it’s had the opposite effect, leading to cost overruns, inefficiencies, and solutions that do not work. Remember the initial rollout of Healthcare.gov? Exactly.

It doesn’t have to be this way.  Believe it or not, there’s a proven solution that has stood the test of time.  In 1977, Lawrence Putnam Sr. discovered the “physics” of how engineers build software by successfully modeling the nonlinear relationship between the five core metrics of software: product size, process productivity, schedule duration, effort and reliability. 

In this article for GCN, QSM's Joe Madden explains how the five core metrics of software estimation make a powerful tool that can be used at each phase of the software acquisition life cycle to help government IT program managers make more objective, quantitative decisions.

Read the full article!

Many of the project managers that I speak with track their software and systems projects at a very detailed level. They use detailed spreadsheets or other tools to track hours and tasks on a daily basis. This is fine, but it's important to manage the big picture so we can avoid assigning detailed tasks to duration and budget goals that are unrealistic.

By "big picture" I mean tracking at the project release level and focusing on a few key actuals: size, duration, effort, reliability, and efficiency. It's important to track these actuals to a reliable plan. These are the measures that can give us the biggest and quickest insight into a project’s potential success or failure. You can see this analysis in the SLIM-Control graphs below, showing the blue plans versus the red actuals.

Once the project is underway and we start tracking the actuals, we can generate new project forecasts based on the actual work delivered, time elapsed, and effort spent. These new forecasts are empirically-based. This will enable us to adapt to change requests, see when the project will finish and how much it will cost. The SLIM-Control graphs below show the blue plans versus the red actuals plus the new forecasts shown in white. SLIM-Control is curve-fitting to the actuals and running empirically-based mathematical models to generate the new forecasts.

The staff build-up plan defines how many, what kind, and when staff are needed for the entire project. Too many or too few, bringing them on too early or late, employing the wrong mix of expertise or experience - avoiding all these pitfalls with a staff build-up plan will ensure a successfully staffed project. Reviewing proposals for a complex project, such as major software development or support, is a challenging activity. Since labor is the major cost and feasibility determinant for such projects, requiring the submission of a "staff build-up plan" and verifying its realism is crucial in determining whether a proposed project can realistically succeed. In this article for Contract Management Magazine, QSM's Gene Shuman identifies the key components of an effective staff build-up plan.

Read the full article!

QSM is a leading demand and vendor management company. We have many years of experience working with outsource management professionals, evaluating software project vendor bids and monitoring the development progress of those bids for our clients. We are often hired to help them with their vendor management process because their past projects have failed to meet cost, effort, reliability, and duration expectations. 

It starts with the independent estimate and bid evaluation process. Our main clients are CIOs, PMO managers, purchasing managers, software project managers, and business stakeholders. Our clients will usually have a large software development or package configuration project pending. They are initially trying to figure out which vendor to hire. Vendor A will offer them a bid of 20 million dollars with a specified duration commitment and Vendor B will offer them a bid of 30 million dollars with a different duration commitment. How do we know which vendor to choose? Can Vendor A really finish with a lower cost and shorter schedule? Is the system going to work when it’s done?

The way it usually works is the client will make a decision based on their experience or gut feel. Or if they have already worked with a specific vendor in the past they will go with that vendor again based on some personal relationships that have evolved. Then the problems start. The work that was promised doesn’t get done within the promised time or the promised budget. The vendor then comes back and says they will add people to the project and everything will be ok. The client approves the revised project plan since they don’t have a way to confirm the accuracy of the revised proposal. Then even bigger problems start. More money is wasted, the schedule slips even more, and relationships sour.

Too many projects are late, over-budget, under-delivered, or a combination.  The problems continue despite widespread awareness and improvements in project management knowledge, tools, and process maturity.  

A recent piece in the Washington Post business section identified a likely culprit: “the planning fallacy”.  Princeton psychologist Daniel Kahneman and Amos Tversky of Stanford describe it as “the tendency to underestimate the time, costs, and risks of future actions and overestimate the benefit of those actions”.  The results are time and cost overruns as well as benefit shortfalls.  The concept is not new: the pair coined the term in the 1970s and has been researching it since.

According to the Post, cognitive biases such as optimism bias (the tendency to expect positive outcomes from one’s actions) and overconfidence can be causes of the planning fallacy. There is a growing body of evidence, collected by researcher Bent Flyvbjerg at Oxford University, that optimism bias is an important bias affecting the quality of forecasts in project planning. 

Other explanations of the fallacy include possible intentional and deliberate considerations on behalf of the planner - such as incentives, organizational pressures and strategic deception. 

There is an old adage that if your only tool is a hammer, everything looks like a nail.  We use the lessons learned and experience we have gained to address current issues.  But if the problem (or software project) we face today is fundamentally different from those we’ve dealt with previously, past experience isn’t the proper framework.  In effect, we will be using a hammer when a saw or a chisel might be the tools we need.

The solution, of course, is to first gain an understanding of the problem at hand.  What are its defining features?  How does it behave?  Only then can a proper solution be designed and the appropriate tools selected.

To a large degree, our understanding of how products are developed comes from knowledge gained from manufacturing since the beginning of the Industrial Revolution.  Mentally, our first instinct is to try to apply those lessons learned to software development.  But there is a huge problem with this approach. The creation of software is not a manufacturing process, but rather a knowledge acquisition and learning process that follows different rules.  Here is a simple example.  If I have an assembly line and want to double my output, I have several choices.  I might add a second shift of workers or I could install an additional assembly line.  Because manufacturing is a repetitive process in which design problems are solved before product construction begins, the relationship between labor required and output remains fairly constant.  In a nutshell, we already know exactly what we need to do (and how to do it).  

How is baseball analysis like software project management?  One way is the ability to continually update estimates and forecasts, as the situation and our knowledge change.  As Larry Putnam Jr recently wrote, “project estimation should continue throughout the entire project lifecycle”. 

Walter Shewhart, the father of Statistical Process Control, explained it like this:

“…since we can make operationally verifiable predictions only in terms of future observations, it follows that with the acquisition of new data, not only may the magnitudes involved in any prediction change, but also our grounds for belief in it.”

Here is a baseball example that should appear very familiar to software estimators who are familiar with the often quoted cone of uncertainty.  The following graph is taken from Curve Ball: Baseball, Statistics, and the Role of Chance in the Game, by Jim Albert and Jay Bennett.

The above model is based upon only a few simple items:  The number of homeruns hit so far; the number of games played so far and number remaining; and the total number of games in a season.  We could try to improve the model, especially early in the season, by incorporating more information.  For example:  

In the software project management field, projects go badly about 43% of the time and fail completely 18% of the time. While there are several reasons for this, and plenty of blame to go around, one of the easiest ways to reduce the risk is to start at the beginning – with the proposal. In a recent interview with Cameron Philipp-Edmonds of StickyMinds, Larry Putnam, Jr. talks about the importance of the proposal when executing a successful project. He identifies five key questions that should be answered before any project starts and how software estimation ties into the proposal process.

Read the full interview transcript here!

Enough already with Healthcare.gov and the many (many) other high-profile IT project failures; let’s talk about government software projects that actually worked. Successful software projects are no accident. Best-in-class government IT projects share common traits that agencies can use to ensure success. In a recent article for Government Computer News, QSM's Larry Putnam, Jr. leverages data from from the QSM Database to identify best practices for successful government projects.

Read the full article!