Ethan Avery's blog

In most organizations, spreadsheets play a vital role for a variety of uses, from time keeping to budgeting to even estimating software development releases.  They offer a level of familiarity - most of us cut our technical teeth on spreadsheets from a young age and have grown with them as we advanced our careers.  When it comes to estimating software development, spreadsheets are a common “go-to,” but an estimation tool offers more flexibility, timely “what-if” changes and far less complexity for new and seasoned estimators alike.

Have you ever been in an organization in which the estimation spreadsheet was so complex and cumbersome that only a select few knew how to use it?  This leaves an organization vulnerable to perhaps a single link of institutional knowledge upon which they are making vital business decisions.  And if that one project manager, analyst or architect leaves the organization, switches roles, or retires, what then?  Even if they go on vacation for 2 weeks, you’re left with a spreadsheet that may have an impossible learning curve for anyone else hence delaying or possibly negating the estimates being created at all.

In talking with many organizations about their IT estimation practices over the years, I’ve noticed a recurring theme has been their self-perception of immaturity when addressing the use of a commercial estimating tool. Estimates in the workforce are typically generated via the Delphi method, multi-tabbed spreadsheets, and uncalibrated guesses.  When recommending a top-down tool approach, the feedback can be: “we aren’t mature enough to use a tool or model.” 

I’ve actually seen the opposite.  The top-down approach offers an on-ramp to more formal estimating by its very nature of needing few inputs, rather than the myriad of cells, rows, and columns required to be populated in a spreadsheet.  A top-down tool approach leads us away from relying on the institutional knowledge of a few people that may have relevant experience, but one day may retire or move on from the organization taking that knowledge -with them.  Also, a parametric top-down approach leveraging relevant historical data is much better than a wet thumb in the air.

During the COVID era, I started thinking of all the home improvement projects I would like to tackle and sadly have not started any, lol.  However, I recently became motivated and decided to apply drywall mud (skim coat) to two walls in my garage that have been dinged and dented over the years.  An essential part of this process is to first estimate my materials, time and effort to minimize the impact of disruption to my family’s day to day lives.  Estimating this job is a simple process – amount of surface area to be covered yields the amount of drywall mud I will need.  The time and effort to complete the project is the fuzziest part of the estimate, but since I have mudded before, I have a rough idea of how long it will take to mud, sand, mud again, sand again, then primer and paint.  The risk of this estimate is low, since my work won’t be keeping a medical device functioning, an airplane afloat or a billing system used by thousands operational.

Estimating software delivery is not that simple and its accuracy can be easily compromised – but what is “accurate?"  Software estimation’s non-linear nature introduces much complexity, since time, effort, resources, and quality are all interdependent - when a change to one of those measures happens it affects the others.

Estimation return on investment - a hotly debated topic in some circles, but still a very active and frequent conversation in C levels all over the world. Something that hasn’t changed in the 20+ years I’ve been involved with estimating software development, regardless of methodology, has been the question asked by senior leadership when embarking on developing new functionality - “How much will it cost and when can we deliver?”  Despite the emergence of new methodologies, this question is understandably the main driver of whether to proceed with a project or not.  The ROI I am focusing on here is the IT budget we are committing to building a project, or portfolio of projects.  Very often, we are faced with that question very early on, when we know little about the project’s full intended capability, but the C level is asking for an estimate and we better respond with a defendable answer.

The ROI is realized in several ways - firstly in the form of estimating the appropriate staffing levels.  We have often assisted clients who initially estimated their projects with too many FTEs.  A relatively simple tweak to the staffing skills combined with empirical data for effort calibration can prove, in many cases, that the FTE count is too high. In the graph below, the yellow diamond represents our current estimate compared to the green dot (adjusted estimate) resting on the blue trendline in the middle representing the industry average for staffing vs. size.  Properly adjusting the FTE levels can save thousands, even hundreds of thousands of dollars depending on the project size, and millions of dollars across a portfolio.

The onset of COVID-19 has revealed many dependent synergies in the world and their mutual effects on one another.  For example, the decrease in human consumption of fossil fuels, not as many cars on the road nor planes in the air has reduced carbon emissions resulting in cleaner air, at least for the time being. 

Another synergistic COVID-19 result affects the IT world, including estimation of software delivery.  Many organizations wisely instilled a work from home (WFH) policy to mitigate the potential spread of disease.  This new work format introduced a fascinating human experiment – can we be as productive en masse during a WFH situation as opposed to being in the office?  The lack of physical proximity stresses the importance of fluid and effective communication.  During pre-COVID-19 days, estimating software delivery was orchestrated through meetings in hallways and conference rooms to communicate project assumptions, constraints and lobbying for project expectations.  But post-COVID-19, we are reliant on remote/virtual contact, limiting the nuances of physical meetings.  Thankfully, there are capabilities in place to make this a fluid process within our SLIM-Collaborate solution. 

Many organizations have a need and interest in introducing tools for estimating their IT development efforts, but are reticent due to the perceived disruption, or all out difficulty of execution.  Estimating touches many parts and processes of the organization, so implementation can seem daunting. However, many organizations I work with are surprised by the relative ease of the integration and how the results pay off in spades.

While a process may have been in place for years, or merely months, any good estimating tool should be able to adapt and be woven into that process.  This includes aligning the estimating tool to the nomenclature of the environment and customizing the tool to an organization’s project types such as Cloud, Agile, ERP, Waterfall etc.  Even a seemingly small step of branding the tool itself to the corporate identity, via company logo placement, can help.  Once a tool has your environmental language and “feel” embedded, it starts to belong.  Initially, all of this can be accomplished at a very centralized level – one or two projects with no need for disrupting the work of the project staff.   

Now that the baseline estimate has been created, and stakeholders feel their inputs and concerns have been addressed, we as purveyors of the estimate have done our job.  In the world of IT project measurement, many organizations will deservedly feel accomplished that they have armed their development staff with an empirically based roadmap from which to navigate the next x number of months toward delivering a product.  Now let the construction and testing begin!  But wait, there’s more!

It’s always wise to have a sound estimate, but for added assurance of hitting the budget, schedule, staffing and risk targets, organizations have the option of tracking the project mid-flight. Just as estimating is conflated with planning, tracking can be equally confused with other one-dimensional monitoring of projects underway.  So many things can change from the time an estimate is created to the time the first iterations are built.  It’s likely that our estimate assumptions will change after some time has passed into the construction process, unless we have reacted to inevitable unforeseen forces.  For example, requirement changes, staff turnover, management demanding the project x weeks/months earlier, but still expecting all the original functionality.  These are all very real events that are thrown at the PM after the project is underway.  We at QSM have provided a solution for this since the mid-80’s in SLIM-Control, a module in our SLIM Suite.

In any development methodology, we throw around the word “estimate” freely not really understanding how it’s interpreted by many.  In many cases, an estimate, regardless of its content and process by which it was created, is received implicitly as a pin point number with the accuracy of multiple decimal points.  This presents a problem for all parties involved.

I recently had a discussion with a gentleman who told me that prior to using our SLIM tool, the estimates in his organization were arrived at by casual hallway conversations, often started with, “how much do you think this will cost, and how long will it take?”  A typical response is, “hmmm, I’d say about 6 months and $500K.”  That innocent musing then becomes the information upon which business decisions are based, leadership bonuses may be won or lost, and the credibility of the dev team is on the line.

I’d strongly recommend adhering to some definitions when talking estimates.  These definitions will help mitigate potential misunderstandings around the agreement of what makes an estimate:

In my daily work life of supporting my clients, I’ve seen the IT project estimating function reside in many different areas of organizations.   Some groups have this function situated in a core group, such as an Estimating Center of Excellence, PMO or Delivery Excellence group.  These teams are responsible for either creating estimates for the organization after being fed some goals/assumptions, or vetting existing estimates born elsewhere to ensure they are in line with the business’ goals without compromising, as best they can, budget, schedule or quality.

Conversely, I’ve worked with organizations in which the project estimation function is supported by organizationally dispersed estimators who are busy with their other tasks.  Decentralized estimators typically serve a sole siloed division with not much interaction outside their team.  Like the bigger shops, these folks are funneled project estimates to be sanity checked and render their analysis with recommendations for adjusting the estimate if it seems out of line. 

Both scenarios above share a common thread – the estimates, regardless of their origin, have an impact on, and are influenced by, a wide breadth of people.  We at QSM call these people stakeholders, contributors, influencers etc.   They all have some kind of investment in the estimate that is important to them.  What if we could involve them in the estimating process?  With more communal agreement on inputs, there won’t be as much rework of the estimate later, when otherwise uninvolved people see their interests aren’t represented.  With everyone feeling they have a say in the estimate, smoother seas are at least more possible.

Managing vendors has become increasingly important in recent years.  In my account management role at QSM, I see both sides of the vendor management relationship.  The client wants a proven vendor that will partner with them in achieving their IT goals; and the vendor wants to win that business, employ their workers, and hopefully earn more work.  Unfortunately, that state of client/vendor Zen is not often achieved, usually due to legitimate (and sometimes not) misunderstandings on both sides.

On the client side, they are concerned with selecting a vendor with whom they are confident their tasks and deliverables will be achieved on time, within budget, and of the best possible quality.  After a round of RFI’s, then RFQ’s, then a final down select process, the vendor is chosen and work begins.  Often, at least in my experience, the overriding decision criteria comes down to cost, which makes sense, to a degree.  But in many cases, cheapest, I mean, least expensive bids often rule the day.  This kind of decision-making comes with its own set of risks; the most obvious is you get what you pay for and it’s often an ill-prepared vendor.