A common misperception is that an estimator’s job is done after a software project’s parameters are set. On the contrary, software estimation should be conducted throughout the project lifecycle to reflect inevitable changes and to improve estimates on other projects. In this article, originally published in Projects at Work, Larry Putnam Jr. identifies three ways to maximize estimating efforts — before, during and after your project is complete.
Since I work for a software metrics and estimation company, many people ask me questions regarding capacity planning and demand management. Most of the project managers that I speak with are using project portfolio management tools for very detailed, task-level resource capacity planning. They spend a lot of time planning the person hours for each task and then these task-level plans are prioritized and viewed across the organization. These are useful tools and methods and they usually require a sizable investment.
The problem is that many of these project managers don’t have a good way to support demand management. That is to say that they aren’t able to accurately estimate the key drivers that go into their PPM tools. They need to be able to answer key questions like: Should we commit to 6 months or 9 for the project duration? Do we need 10 software developers or 20 to finish in 9 months? How much is the overall project going to cost? What alternatives do we have? Has anyone ever achieved that duration in the past? Should we even go forward with this project; what is the risk?
Oftentimes the project manager comes up with this information informally based on experience. Unfortunately, when they don’t use a scientific approach to estimating, they leave out key factors that affect the estimate and project success, like project complexity, team efficiency, and overall project uncertainty.
Client organizations who are considering investing in SLIM® (a top-down, scope-based, parametric software estimation tool) often ask us for return on investment (ROI) case study examples which we gladly provide to help them with their business case. However, one question that has never been asked but I have always wondered is: does ROI accelerate with increased investment or does it follow the law of diminishing returns?
To answer this question, we looked at seven software estimation ROI case studies that included a variety of small, medium and large clients, from a single seat of SLIM all the way up to an enterprise Estimation Center of Excellence (ECoE).
On the above chart we plotted the investment in SLIM tools and consulting on the X axis vs. the return (actual cost savings) on the Y axis for each case study using a logarithmic scale. We then drew a trend line through the data points.
Following the trend line, small engagements (~$30K) had an average ROI of more than 13:1. Medium engagements (~$300K) had an average ROI of 33:1. Large ECoE engagements ($3M) had an average ROI of 67:1. So not only is the ROI compelling, but it also accelerates with increased investment.
The actual cost savings (return) as reported by, or observed while working with, our clients include:
I am a professional software project estimator. While not blessed with genius, I have put in sufficient time that by Malcolm Gladwell’s 10,000 hour rule, I have paid my dues to be competent. In the past 19 years I have estimated over 2,000 different software projects. For many of these, the critical inputs were provided and all I had to do was the modeling. Other times, I did all of the leg work, too: estimating software size, determining critical constraints, and gathering organizational history to benchmark against. One observation I have taken from years of experience is that there is overwhelming pressure to be more precise with our estimates than can be supported by the data.
In 2010 I attended a software conference in Brasil. As an exercise, the participants were asked to estimate the numerical ranges into which 10 items would fall. The items were such disparate things as the length of coastline in the United States, the gross domestic product of Germany, and the square kilometers in the country of Mali: not things a trivia expert would be likely to know off hand. Of 150 participants, one person made all of the ranges wide enough. One other person (me) got 9 out of 10 correct.
The laws of flight have many commonalities with the “physics” of software development. Whereas the principles of aerodynamics are essentially about thrust, drag, lift and weight, developing software has always been about the relationship between size (scope), effort (cost), duration (schedule) and the often overlooked measurement of quality (reliability).
In aviation parlance the expression “flying by the seat of your pants” literally meant that you felt the plane through your “seat”. Early aircraft had limited navigational aids, mainly a rudimentary compass (which worked only when flying level) and a simple string to assess airflow relative to the plane’s fuselage. Pilots could determine the degree of an ascent or descent by G-force sensation, or assess airspeed by the severity of the aircraft’s vibrations. Until the invention of the gyroscope it was quite dangerous to fly without a virtual horizon – the centerpiece of a modern dashboard.
Early aeronauts plotted their routes using individual skill, celestial or fixed landmarks and their own real time perceptions rather than depending on mechanical tools. However, fog or low visibility would render the limited instruments they had useless, with the flight itself remaining as a risky endeavor. A successful trip meant you landed at your destination in one piece, but it was largely dependent upon the talent and judgment of the aviator, visibility and weather, and perhaps no small amount of luck.
I’m a developer in our IT department and we know that project estimating is a big deal for our customers. Somehow, no matter what we do, we can't seem to get it right. We do know that project size is an important input to good estimating and our gut feel is that if we get sizing right, we’ll do better estimates! I know you recommend using function points, but I’ve also been reading a lot about use case points, story points, SLOC, sizing by analogy, T-shirt sizing, COSMIC and other sizing metrics. We do a mix of waterfall, agile, iterative and even Kanban to do our projects so what’s the best choice for sizing to get the best results?
- Size Challenged in Milwaukee
Dear Size Challenged:
Sometimes I wonder if the internet and the proliferation of (mis)information is a good thing. Before the internet, our choices (for sizing or estimating or anything) were limited and we didn’t have such an overwhelming task to first sift through many options before taking action. Your list of software sizing choices is an example of this.
Presented by Laura Zuber.
A business stakeholder (project manager, account representative, etc.) is faced with an all-too-familiar challenge: his client requests a quote for developing a new application within a very short timeframe, and not much information about scope to go on. In this webinar, QSM's Laura Zuber shows how the business stakeholder can produce a Rough Order of Magnitude (ROM) estimate on the spot, using QSM's web-based solution, SLIM-WebServices. Follow his process as he collaborates with corporate estimation specialists to refine those initial estimates as they advance to more detailed stages. By incorporating contributions of a variety of project stakeholders in the estimation analysis process, the business stakeholder is able to make better business decisions.
Laura Zuber has over 22 years of experience in software development consulting and training, nine of which have been with QSM. She conducts training and demonstrations for all QSM SLIM Suite Tools and serves as a Lead QSM Support Representative. Prior to coming to QSM, Laura managed software development projects, and served as a senior software process improvement specialist at SAIC. She has performed process assessments, designed and implemented best practices, and co-lead the corporate metrics training program.
In a recent, highly-discussed article for Dr. Dobb's, QSM's Carol Dekkers asks a tough question: why are we so woefully poor at estimating software projects? It's a tough pill to swallow considering software developers are among the smartest people on the planet, often boasting advanced degrees in mathematics, engineering, or computer science. Yet study upon study cites that less than one-third of projects are delivered on time or on budget. The problem of software project estimation is not straightforward. To get the heart of the issue, Carol Dekkers takes us through the five top misperceptions about software estimating, and what we can do to address them.
Read the full article on Dr. Dobb's!
I think it’s safe to say that nobody really enjoys hearing bad news. It’s especially hard if you’re the person who has to deliver the bad news, particularly to a superior. How will your boss react? Will you be the one held responsible (unfairly) for the project failure? These are all reasons for keeping the ‘bad news’ to yourself and letting those in charge find out on their own.
I’ll share a story about one of the first jobs I ever held, as an assistant manager at a summer swimming pool. My supervisor had a very hands-off approach to management and would often rely on me and the other assistant managers to handle the day-to-day operations of the pool. Whenever I would deliver less-than-favorable news to him, such as our pool vacuum breaking, or a health inspector dropping by to schedule a visit, my supervisor would literally stick his fingers in his ears and say “La la la la la, I can’t hear you. Taylor, you know how I feel about bad news. Fix the problem.” This put me in a very awkward situation, because as a high school student, I didn’t necessarily have the training or the authority to fix every problem myself, in order to shield him from the ‘bad news.’
Unfortunately, this type of management exists beyond the pool house and can frequently be found in the corporate world as well. In an environment where your reputation can mean everything, stakeholders can be very reluctant to receive bad news about the status of their project. The silver lining in this is that receiving ‘bad news’ isn’t necessarily always a bad thing. Allow me to explain.
I work in IT and we do a lot of our software development projects based on pre-defined delivery dates (no one really knows where they come from). Sometimes this works out, but usually we end up delivering the project months late because we had no idea the project was as big or as complex as anyone had originally thought. A friend says his company uses “t-shirt sizing” for project estimates and I’ve never heard of it. What can you tell me about this new approach?
- I’m willing to learn from the fashion industry
Dear I’m willing:
T-shirt sizing has been around in one form or another for a few years but it is not a widespread mainstream estimating method. It is a quick and dirty way to estimate software size using ranges of size. Once you have the relative size (e.g., XS, Small, Medium, Large, XL, XXL, or larger), you can enter it into several different estimating tools as the size on which to base the estimate. While this doesn’t give you a guaranteed size for the software to be delivered, it is better than not having any idea of the size. Size (scope) is one of the three main tenets of the triple constraints (scope, budget and schedule) for project management. Given one, you can estimate the other two.