"The game ain't over 'til it's over." - Yogi Berra

Baseball season is here and with apologies to the late Mr. Yogi Berra, “it’s like déjà vu all over again.”

Why would a project team or program management office (PMO) take the time and spend the resources to collect information about a project that was just completed? Isn’t this the time when victory is declared and everyone runs for the hills? In many cases, delving into what happened and what actual costs and durations were incurred can seem like an exercise in self-flagellation.

Historical data is arguably the most valuable input available in the software estimation process. While other inputs such as size and available duration or staffing can be seen as constraints, properly collected historical data moves the activity from the realm of estimating closer to “factimating.”

One of the things I hear from many c-level managers is how difficult it is and how long it takes to generate reliable resource plans at the enterprise level. Many organizations take months to generate their annual budgets and often times the negotiated budgets end up being unrealistic. To fix this problem we need to combine good capacity planning with good demand management. There are a number of project portfolio management tools to help with the capacity planning. The problem is the numbers will be off if we don’t get the demand management part right.

There are world class demand management tools available that can be used by business decision makers. These tools allow us to come up with empirically based, reliable project and enterprise level resource plans. In the SLIM-Estimate view below you can see the forecasted effort by role by month.

The view below shows the plan being sanity checked with industry data so we can better negotiate our budgets with confidence.

The even bigger news is that we can automatically feed our empirically based demand numbers into our PPM tools, making the job of capacity planning much easier and more reliable. In the view below you can see two separate plans, represented in side by side columns. The original PPM plan was updated automatically from an empirically based and sanity checked plan from SLIM-Estimate.

Outsourcing was supposed to make government IT executives’ lives easier. Yet in too many cases, it’s had the opposite effect, leading to cost overruns, inefficiencies, and solutions that do not work. Remember the initial rollout of Healthcare.gov? Exactly.

It doesn’t have to be this way.  Believe it or not, there’s a proven solution that has stood the test of time.  In 1977, Lawrence Putnam Sr. discovered the “physics” of how engineers build software by successfully modeling the nonlinear relationship between the five core metrics of software: product size, process productivity, schedule duration, effort and reliability. 

In this article for GCN, QSM's Joe Madden explains how the five core metrics of software estimation make a powerful tool that can be used at each phase of the software acquisition life cycle to help government IT program managers make more objective, quantitative decisions.

Read the full article!

The story of QSM and software application estimation begins during my time in the Army. I was assigned to Sandia Base, NM to research methods for protecting soldiers from the effects of nuclear explosions.  I had to do several calculations to determine the impact of an explosion (blast calculations) on soldiers using a slide rule, which was very tedious.  Sandia National Laboratory was next door to my office, and they had just gotten the biggest and best engineering computer available at the time.  They offered computer time for anyone needing it and even offered to teach me programming, so I decided to take a course in FORTRAN programming over my lunch hour so I could do my blast calculations quicker. These lessons aided me in completing my work at Sandia and followed me to my future assignment at the Pentagon. 

For my tour at the Pentagon in the 1970s, there was not a lot of need for my nuclear experience so I was assigned to the Army’s computer program. We had to defend our program budget to the Department of Defense (DoD) budget review authority (OSD). One system, SIDPERS, the Army enterprise personnel system, had been in development for five years and after having a peak staff of 110, we were projecting 93 people for the next five years. The analyst looking at the budget asked what should have been a simple question, “What are these people going to do?” I did not have a good answer, and later, going back to the project team, neither did they. Because of this we lost $10M in our budget.

QSM's C. Taylor Putnam-Majarian and Doug Putnam recently published an article, Understanding Quality and Reliability, on InfoQ.

One of the most overlooked but important areas of software estimation, measurement, and assessment, is quality. It often is not considered or even discussed during the early planning stages of all development projects, but it’s almost always the ultimate criteria for when a product is ready to ship or deploy. Therefore, it needs to be part of the expectation-setting conversation from the outset of the project. So, how can we talk about product quality? It can be measured a number of ways, but two in particular give excellent insights into the stability of the product.

Read the full article on InfoQ!

Often I speak with IT project measurement folks about the permeation of agile into their project estimation processes.  While the Agile Manifesto recently celebrated its 15^th birthday, it’s just been the last several years that I’ve seen agile gain substantial momentum in becoming the official method many companies shepherd their projects.  Or has it…?

The graphic above shows that the use of agile over waterfall has yielded desirable results in terms of reaching business goals of delivering on time, within budget and with complete functionality.  Yet I still hear IT project measurement people voice concerns about fully adopting agile for their projects.

My observations, from talking to many people trying to estimate agile projects, are from a purely estimating perspective.  Before agile emerged, many development shops were building their projects via waterfall, ERP, RUP and other methods, all of which provided respective value.  Upon agile’s debut, it was but a whisper, among a few, as the next leading development methodology.  Over the years skeptical minds were slowly changed and adoption increased.  Today more organizations are using agile to develop their projects, but I have found a surprising number of estimators telling me in a hushed tone – they aren’t really developing in agile, despite the proclamation from on high.  They are using more a combination of agile and other development methods, yielding an agile hybrid, sometimes sheepishly referred to as “wagile” or “agilefall”.

The time has come, once again for QSM’s annual March Madness tournament.  As we enter our 6^th year of friendly office competition, I looked back at some of my previous strategies to help me figure out how I wanted to go about completing my bracket this year.  In doing this, I realized that many of these concepts can be applied towards IT project management.

Three years ago, I built my bracket around an emotional desire for my preferred team to win.  I paid very little attention to their previous performance that season, or any of the other teams for that matter.  Needless to say, I did not do as well as I had hoped that year.  Unfortunately, this strategy is applied fairly frequently in software estimation, with stakeholders committing their teams to unreasonable schedules and budgets for projects that are “too big to fail.”  Committing to a plan based off of the desired outcome does not produce a good estimate, and often results in cost overruns and schedule delays (or in my case, quite a bit of ridicule from the Commish).

I recently came across this blog post by David Gordon and I believe it nicely summarizes the problem many C-level executives find with the #NoEstimates agile view. At the end of the day, they need realistic numbers in order for their organizations to make a profit and remain competitive in their markets. It's simply not enough to start development without some sort of upfront plan as to how much the overall project will cost. Gordon gives a nice analogy:

Now, picture a carpenter who has been asked to bid on framing a group of houses. He replies that he can’t tell how long it will take, because his crew hasn’t built that model before. But if the home builder gives them, say, $400,000, they will work diligently, and when the money runs out, they can decide together whether to put more money into the construction. You’re probably thinking that the builder won’t ever do business with that carpenter, and neither will anyone else in town. But that’s because the carpenter isn’t an employee—he has competition. This helps explain why certain software developers think #NoEstimates is a fine idea—they don’t perceive that they have competition.

Unfortunately for developers, this is not a sustainable way of operating - as many organizations continue to outsource more and more positions, developers will need to be able to justify their work as a business case. Gordon argues it's worthwhile for developers to learn software estimating best practices and to "...take some time away from learning the newest cool development tool to become viable as a contingent worker." I have to say that I agree.

In software estimation, some discovered relationships turn out to be true primary principals of software development.

Way back in 1978, Larry Putnam, Sr. discovered that the relationship between project duration and project effort was exponential.¹  His equation equated to:

Duration in months = 2.15 times the cube root of effort in manmonths

In his 1981 book, Barry Boehm described the nominal relationship in COCOMO² as:

Duration in months = 2.5 times the cube root of effort in manmonths

Very similar results.  Is that something specific to the way projects were managed way back then?  Or, is this a true fundamental law of software project management?

Sometimes, it is fun and also informative to revisit pioneering work to see how things have (or have not!) changed over the decades since.  I have used updated benchmarking data to check this staffing relationship and found it to be surprisingly persistent. 

I took project Main Build (Design through Test) effort and Main Build duration from the QSM database, for projects that have competed in the 21^st century.

The following graph has duration in months on y axis, and effort in personmonths on x axis.

The exponential regression shows that the “nominal” duration of these projects = 2.0 x cubed root of effort.  

Many of the project managers that I speak with track their software and systems projects at a very detailed level. They use detailed spreadsheets or other tools to track hours and tasks on a daily basis. This is fine, but it's important to manage the big picture so we can avoid assigning detailed tasks to duration and budget goals that are unrealistic.

By "big picture" I mean tracking at the project release level and focusing on a few key actuals: size, duration, effort, reliability, and efficiency. It's important to track these actuals to a reliable plan. These are the measures that can give us the biggest and quickest insight into a project’s potential success or failure. You can see this analysis in the SLIM-Control graphs below, showing the blue plans versus the red actuals.

Once the project is underway and we start tracking the actuals, we can generate new project forecasts based on the actual work delivered, time elapsed, and effort spent. These new forecasts are empirically-based. This will enable us to adapt to change requests, see when the project will finish and how much it will cost. The SLIM-Control graphs below show the blue plans versus the red actuals plus the new forecasts shown in white. SLIM-Control is curve-fitting to the actuals and running empirically-based mathematical models to generate the new forecasts.