Estimation return on investment - a hotly debated topic in some circles, but still a very active and frequent conversation in C levels all over the world. Something that hasn’t changed in the 20+ years I’ve been involved with estimating software development, regardless of methodology, has been the question asked by senior leadership when embarking on developing new functionality - “How much will it cost and when can we deliver?”  Despite the emergence of new methodologies, this question is understandably the main driver of whether to proceed with a project or not.  The ROI I am focusing on here is the IT budget we are committing to building a project, or portfolio of projects.  Very often, we are faced with that question very early on, when we know little about the project’s full intended capability, but the C level is asking for an estimate and we better respond with a defendable answer.

The ROI is realized in several ways - firstly in the form of estimating the appropriate staffing levels.  We have often assisted clients who initially estimated their projects with too many FTEs.  A relatively simple tweak to the staffing skills combined with empirical data for effort calibration can prove, in many cases, that the FTE count is too high. In the graph below, the yellow diamond represents our current estimate compared to the green dot (adjusted estimate) resting on the blue trendline in the middle representing the industry average for staffing vs. size.  Properly adjusting the FTE levels can save thousands, even hundreds of thousands of dollars depending on the project size, and millions of dollars across a portfolio.

The onset of COVID-19 has revealed many dependent synergies in the world and their mutual effects on one another.  For example, the decrease in human consumption of fossil fuels, not as many cars on the road nor planes in the air has reduced carbon emissions resulting in cleaner air, at least for the time being. 

Another synergistic COVID-19 result affects the IT world, including estimation of software delivery.  Many organizations wisely instilled a work from home (WFH) policy to mitigate the potential spread of disease.  This new work format introduced a fascinating human experiment – can we be as productive en masse during a WFH situation as opposed to being in the office?  The lack of physical proximity stresses the importance of fluid and effective communication.  During pre-COVID-19 days, estimating software delivery was orchestrated through meetings in hallways and conference rooms to communicate project assumptions, constraints and lobbying for project expectations.  But post-COVID-19, we are reliant on remote/virtual contact, limiting the nuances of physical meetings.  Thankfully, there are capabilities in place to make this a fluid process within our SLIM-Collaborate solution. 

Many organizations have a need and interest in introducing tools for estimating their IT development efforts, but are reticent due to the perceived disruption, or all out difficulty of execution.  Estimating touches many parts and processes of the organization, so implementation can seem daunting. However, many organizations I work with are surprised by the relative ease of the integration and how the results pay off in spades.

While a process may have been in place for years, or merely months, any good estimating tool should be able to adapt and be woven into that process.  This includes aligning the estimating tool to the nomenclature of the environment and customizing the tool to an organization’s project types such as Cloud, Agile, ERP, Waterfall etc.  Even a seemingly small step of branding the tool itself to the corporate identity, via company logo placement, can help.  Once a tool has your environmental language and “feel” embedded, it starts to belong.  Initially, all of this can be accomplished at a very centralized level – one or two projects with no need for disrupting the work of the project staff.   

Now that the baseline estimate has been created, and stakeholders feel their inputs and concerns have been addressed, we as purveyors of the estimate have done our job.  In the world of IT project measurement, many organizations will deservedly feel accomplished that they have armed their development staff with an empirically based roadmap from which to navigate the next x number of months toward delivering a product.  Now let the construction and testing begin!  But wait, there’s more!

It’s always wise to have a sound estimate, but for added assurance of hitting the budget, schedule, staffing and risk targets, organizations have the option of tracking the project mid-flight. Just as estimating is conflated with planning, tracking can be equally confused with other one-dimensional monitoring of projects underway.  So many things can change from the time an estimate is created to the time the first iterations are built.  It’s likely that our estimate assumptions will change after some time has passed into the construction process, unless we have reacted to inevitable unforeseen forces.  For example, requirement changes, staff turnover, management demanding the project x weeks/months earlier, but still expecting all the original functionality.  These are all very real events that are thrown at the PM after the project is underway.  We at QSM have provided a solution for this since the mid-80’s in SLIM-Control, a module in our SLIM Suite.

In any development methodology, we throw around the word “estimate” freely not really understanding how it’s interpreted by many.  In many cases, an estimate, regardless of its content and process by which it was created, is received implicitly as a pin point number with the accuracy of multiple decimal points.  This presents a problem for all parties involved.

I recently had a discussion with a gentleman who told me that prior to using our SLIM tool, the estimates in his organization were arrived at by casual hallway conversations, often started with, “how much do you think this will cost, and how long will it take?”  A typical response is, “hmmm, I’d say about 6 months and $500K.”  That innocent musing then becomes the information upon which business decisions are based, leadership bonuses may be won or lost, and the credibility of the dev team is on the line.

I’d strongly recommend adhering to some definitions when talking estimates.  These definitions will help mitigate potential misunderstandings around the agreement of what makes an estimate:

In my daily work life of supporting my clients, I’ve seen the IT project estimating function reside in many different areas of organizations.   Some groups have this function situated in a core group, such as an Estimating Center of Excellence, PMO or Delivery Excellence group.  These teams are responsible for either creating estimates for the organization after being fed some goals/assumptions, or vetting existing estimates born elsewhere to ensure they are in line with the business’ goals without compromising, as best they can, budget, schedule or quality.

Conversely, I’ve worked with organizations in which the project estimation function is supported by organizationally dispersed estimators who are busy with their other tasks.  Decentralized estimators typically serve a sole siloed division with not much interaction outside their team.  Like the bigger shops, these folks are funneled project estimates to be sanity checked and render their analysis with recommendations for adjusting the estimate if it seems out of line. 

Both scenarios above share a common thread – the estimates, regardless of their origin, have an impact on, and are influenced by, a wide breadth of people.  We at QSM call these people stakeholders, contributors, influencers etc.   They all have some kind of investment in the estimate that is important to them.  What if we could involve them in the estimating process?  With more communal agreement on inputs, there won’t be as much rework of the estimate later, when otherwise uninvolved people see their interests aren’t represented.  With everyone feeling they have a say in the estimate, smoother seas are at least more possible.

Managing vendors has become increasingly important in recent years.  In my account management role at QSM, I see both sides of the vendor management relationship.  The client wants a proven vendor that will partner with them in achieving their IT goals; and the vendor wants to win that business, employ their workers, and hopefully earn more work.  Unfortunately, that state of client/vendor Zen is not often achieved, usually due to legitimate (and sometimes not) misunderstandings on both sides.

On the client side, they are concerned with selecting a vendor with whom they are confident their tasks and deliverables will be achieved on time, within budget, and of the best possible quality.  After a round of RFI’s, then RFQ’s, then a final down select process, the vendor is chosen and work begins.  Often, at least in my experience, the overriding decision criteria comes down to cost, which makes sense, to a degree.  But in many cases, cheapest, I mean, least expensive bids often rule the day.  This kind of decision-making comes with its own set of risks; the most obvious is you get what you pay for and it’s often an ill-prepared vendor.

“No, we don’t have any historical project data collected” is the statement I hear with some frequency when speaking to organizations about their IT project estimating processes.  Ideally we use client history to calibrate and tune the project estimates we provide.  In my quest to spread the word about parametric estimating I often encounter this notion that organizations don’t believe they have historical data in a retrievable form.  In almost every case that I have been involved, it turned out that the historical data was present, just not in the form of a 1,000 rowed spreadsheet.  Often times the data is more available than the client is aware.

Our approach works at a macro level so we are seeking overall project metrics of cost, schedule, size, staffing and defects.  If the actual formal documentation of history is not available for these five core metrics, then it usually is available by leveraging various sources within the organization.  We have found it’s common to resurrect a project’s outcome by seeking feedback from the team that worked the project, however if that’s not possible due to attrition, re-org or other disrupting factors, we can usually find the project metrics through other means.  Those other means may be time and defect tracking tools, requirements analysis tools and accounting systems.  The data is almost always documented somewhere.   

Recently a friend of mine sent me a link to a YouTube video featuring Jeff Sutherland and Ken Schwaber, the founders of Scrum, discussing the latest update to the 2016 Scrum Guide.  The updates to the guide were comprised of survey feedback from the Scrum community, with the goal to understand what is important to them that should be included in the updated Scrum guide.  

The most compelling part of this discussion for anyone contemplating whether estimating belongs in Scrum happens at the 32:10 point.  Jeff and Ken are posed the question, “what is the difference between a forecast and an estimate?”   They answer with the idea that many people confuse the word estimate with commitment, which plagues any development effort regardless of method.  Subsequently Jeff and Ken changed the word “estimate” to “forecast” in the latest Scrum Guide to reflect the ever-changing growth and reduction in project scope, i.e., the estimate will show our best commitment and what we think is going to happen.  That can change once the project starts, BUT, we can measure that too through forecasting, essentially re-estimating throughout the project.  

Recently the correlation between seeking the best gasoline prices and software project overspending collided for me.  On one hand, IT projects will easily outspend their budgets, and on the other hand, in our private lives, we are doing all we can to save pennies in our personal budgets.

From time to time I have found myself pondering whether to drive the extra 5 miles out of my way in order to beat the system and enjoy the best gasoline price in my area.  I don’t believe I am alone in this quest as evidenced by the websites that publish the lowest gas prices within a certain radius.  Typically the competing stations have very small differences between their prices.  If I play my cards right, I may be able to save a total of $.50 - $.75 for a fill up.  In almost all other walks of life $.75 is not even worth a second thought.