Updated Performance Benchmark Table

The latest version of QSM’s Performance Benchmark Table is live!

QSM is excited to announce the release of their latest version of the Performance Benchmark Table.  Last updated in 2009, the table provides a high-level reference for benchmarking and estimating IT, Engineering, and Real Time Systems.  It displays industry average duration, effort, staff, and SLOC (or FP) per Person Month for the full range of project sizes encompassed by each trend group. 

The results were analyzed from a database of 1,115 high or moderate confidence projects completed between 2008 and 2012.  Sixteen countries and 52 different languages were represented in this sample.  In addition to the industry average, minimum and maximum values were also provided for each metric to help give a range of possible results.

The project sizes differed somewhat from the previous version to accommodate the new range of sizes present in the data.  Rather than using the same project sizes across trend groups, we selected project sizes specific to each trend.  Since Business projects are typically smaller than Engineering or Real Time projects, this allows readers to select a size relevant to the type of project they’re estimating or benchmarking.  

This tool can be particularly useful to developers and/ or project managers who are new to estimation or do not have historical project data.  

Blog Post Categories 
Benchmarking Estimation

Webinar - Building an Estimation Center of Excellence

On Thursday, June 13, at 1:00 PM EDT, Larry Putnam, Jr. will present Building an Estimation Center of Excellence.

The pressure to succeed in software development is higher than ever - the current economic climate demands we do more with less, there is fierce global and domestic competition, time-to-market expectations are high, and your company's reputation is on the line. When projects fail, the failure to meet expectations is more often an estimation or business decision failure than a production or execution issue. In this webinar, industry expert Larry Putnam, Jr. takes you through the key elements and step-by-step process for setting up an estimation center of excellence that will ensure your projects succeed.

Larry Putnam, Jr. has 25 years of experience using the Putnam-SLIM Methodology. He has participated in hundreds of estimation and oversight service engagements, and is responsible for product management of the SLIM Suite of software measurement tools and customer care programs. Since becoming Co-CEO, Larry has built QSM's capabilities in sales, customer support, product requirements and most recently in creating a world class consulting organization. Larry has delivered numerous speeches at conferences on software estimation and measurement, and has trained - over a five-year period - more than 1,000 software professionals on industry best practice measurement, estimation and control techniques and in the use of the SLIM Suite.

Watch the replay!

Blog Post Categories 
Webinars Estimation

Estimating for the Business Plan

Having worked in sales and customer service at QSM for over 17 years, I speak to hundreds of professionals each year that are directly or indirectly involved with software development projects using many different development processes. One of the things that I hear from time to time is that estimating is not as important when working with more iterative development methodologies. Some of the reasons I hear most often are that “team sizes are smaller,” “work can be deferred until the next iteration,” “we are different,” and “we are agile.”

As I dig deeper though, I find that the fundamental questions that software estimates answer are relevant, no matter what development methodology is being used. Before committing to a project, executives and managers need to determine reasonable cost, schedule, and how much they can deliver. This is when not very much is known and before any detailed planning occurs.  Estimating helps mitigate risk early in the project lifecycle. Companies also need to have reliable information in order to negotiate with clients. How can we negotiate a schedule and a budget on any project without a defensible estimate? 

When looking at QSM research based on our database of over 10,000 industry projects, a common theme that we see in failed projects is that development team performance is often not the issue. When it comes to missed schedules and budgets, many of the problems occur when expectations are too high and when estimates are not a priority. If we don’t have a reliable estimate up front before the project starts, it’s tough to plan ahead. 

Blog Post Categories 

What Basketball Can Teach Us About Software Estimation

I discovered early on that the player who learned the fundamentals of basketball is going to have a much better chance of succeeding and rising through the levels of competition than the player who was content to do things his own way. A player should be interested in learning why things are done a certain way. The reasons behind the teaching often go a long way to helping develop the skill. — John Wooden

John Wooden is regarded as one of the greatest college basketball coaches. He believed that after talent, courage, and character, fundamentals built successful teams. Successful software projects result from knowing and practicing fundamentals as well, and it begins with estimation.

I thought it would be fun to see if Coach Wooden, by way of noted quotes, could help simplify a few SLIM core concepts.

John Wooden

SLIM Core Concepts

"Don't let what you cannot do interfere with what you can do."

Estimates are uncertain. The accuracy of your estimate depends on the detail and relevance of the data upon which it is derived. Do not succumb to “paralysis by analysis.” You cannot commit to a detailed estimate early in the life cycle because you simply do not have the data to support it.  What you can do is generate a reasonable expected result within a range of potential outcomes based upon industry data or your past performance. The estimate will be good enough to allow data-driven decisions and negotiations.  You can improve the estimate as soon as more detailed information is available.

Blog Post Categories 

Does Your Estimate Accurately Reflect the Five Dimensions of Software Trade-offs?

A recent series of posts by Karl Wiegers eloquently discusses the "reality of tradeoffs" software professionals deal with every day, going beyond the typical success drivers (time, cost, and quality) to include product features and staff. He shares inspiring practical information by making distinctions between constraints, drivers, and degrees of freedom, each representing the amount of flexibility the project manager has to adjust a key factor.

SLIM-Estimate has modeled the non-linear interdependencies of these metrics for over thirty years. It accounts for Wiegers’s five project success factors explicitly, showing the tradeoffs between them in real time. I have mapped Wiegers’s Five Dimensions to SLIM-Estimate’s parameters to show how you can use SLIM-Estimate quantify the trade-offs Wiegers describes.

Blog Post Categories 
Estimation Tips & Tricks

Process and Tools Together

At QSM we offer Estimation Process Consulting Services and the SLIM-Estimate tool. In my 16 years at QSM, I have probably spoken with hundreds of project managers about the pain that they have in the estimation area. Many tell me that they want to finish implementing their process before they bring in a tool.

One of the things that I have learned over the years is that it can be extremely beneficial to bring in the tool while the process is being developed. A successful estimation process implementation is about getting the right project data in the right place for consistent collaboration and results. Implement both the process and the tool at the same time and you can save a ton of money in rework costs down the road. 

A good estimation tool provides a roadmap and a communication vehicle for a successful estimation process. The tool collects the core metrics that the process requires. It also streamlines the results to give the user exactly what they need to know: risk, size, duration, effort, reliability, and productivity.

Some will spend years writing and implementing their process. Why not get the estimation tool in place sooner to make things easier?

Blog Post Categories 

For More Accurate Software Estimates, Avoid Hidden Risk Buffers

A colleague of mine recently sent me a blog post explaining the difference between project contingency and padding.  The blogger made the distinction that padding is what often gets added to an individual’s estimate of the effort required to perform a task (in her example, a software development task) to account for project ‘unknowns’.  The estimator determines the most likely required effort, then pads it with a little more effort in order to arrive at an estimate to which he or she can commit.  Thus, padding represents an undisclosed effort reserve (and implied schedule reserve) to buffer against potential risk.  Contingency reserve, she explains, is “an amount of money in the budget or time in the schedule seen and approved by management.  It is documented.  It is measured and therefore managed.”  Ms. Brockmeier is correct in promoting contingency as the better management tool.  The challenge is having a method to measure and document this contingency and the known unknowns it is buffering.

Implicit Risk Buffer

Padding is a natural result of bottoms-up, effort-based estimation techniques.  Estimating low-level WBS elements creates more opportunity for padding, because the number of unknowns grows with the task list.  The estimator is consciously or unconsciously assessing the risk of each task, considering its dependencies and complexities.  What is being implied in the effort estimate is: 1] an assessment of product size and complexity, and 2] a productivity valuation.

Blog Post Categories 
Risk Management Estimation

Webinar Replay Now Available: Successful Estimating Processes Using the SLIM API

If you were unable to attend our most recent webinar, Successful Estimating Processes Using the SLIM API, a replay is now available.

How do best in class development organizations achieve maximum return on investment from their estimation programs? By leveraging the SLIM API for integrations between estimation tools and detail-oriented products, development teams are able to simplify estimation processes and broaden the estimation program user base. Presented by Carl Engel of IBM Global Services, Scott Lancaster of State Street, and Larry Putnam, Jr. of QSM, this webinar explores two successful implementations of the SLIM API between third party tools and the SLIM Suite. 

Carl Engel is the Estimating Program Manager for IBM's Global Business Services responsible for the development and deployment of performance benchmarking and estimating process, methods and tools including the support for nearly 1,000 SLIM Suite users. Carl has been with IBM for 12 years as an Associate Partner and has had previous roles as the program manager for IBM's project management methodology and tools. He is an IBM certified Executive Project Manager, PMP with over 30 years of program and project management experience primarily in very large scale efforts in the nuclear industry and U.S. National Laboratories.

Blog Post Categories 
Webinars Estimation

Software Cost Estimation Article in The DACS Journal

The February issue of the DACS Journal of Software Technology focuses on Software Cost Estimation and Systems Acquisition. My contribution, which you can read here, addresses the challenges faced by estimators and the value of establishing a historical baseline to support smarter planning, counter unrealistic expectations, and maximize productivity.

Using several recent studies, my paper addresses the following questions:

  • What is estimation accuracy, and how important is it really?
  • What is the connection between the Financial Crisis of 2008 and software estimation?
  • Why do small team projects outperform large team projects?
  • How can you find the optimal team size for your project?

Read the full article.

Blog Post Categories 
Estimation Articles

Part III: The Caveats

In Part 1 of How Much Estimation? we noted that there is an optimal amount of time and effort that should be spent in producing an estimate based on the target cost of a project and business practice being supported.

In Part 2: Estimate the Estimate, we saw that the formula to calculate this optimal time (as measured at NASA)  calculates the Cost of Estimate as the Target_Cost raised to the power 0.35 (approximately the cube root of the Target Cost).  The factor that defines the business practice (either by early lifecycle phase or perhaps by the “expected precision” of the estimate) is a linear factor ranging from a value of 24 to a value of 115.

Those Caveats!

I mentioned that there were caveats with the calculation.  Here they are:

Blog Post Categories 
Estimation SLIM-Estimate