Now that the baseline estimate has been created, and stakeholders feel their inputs and concerns have been addressed, we as purveyors of the estimate have done our job.  In the world of IT project measurement, many organizations will deservedly feel accomplished that they have armed their development staff with an empirically based roadmap from which to navigate the next x number of months toward delivering a product.  Now let the construction and testing begin!  But wait, there’s more!

It’s always wise to have a sound estimate, but for added assurance of hitting the budget, schedule, staffing and risk targets, organizations have the option of tracking the project mid-flight. Just as estimating is conflated with planning, tracking can be equally confused with other one-dimensional monitoring of projects underway.  So many things can change from the time an estimate is created to the time the first iterations are built.  It’s likely that our estimate assumptions will change after some time has passed into the construction process, unless we have reacted to inevitable unforeseen forces.  For example, requirement changes, staff turnover, management demanding the project x weeks/months earlier, but still expecting all the original functionality.  These are all very real events that are thrown at the PM after the project is underway.  We at QSM have provided a solution for this since the mid-80’s in SLIM-Control, a module in our SLIM Suite.

In any development methodology, we throw around the word “estimate” freely not really understanding how it’s interpreted by many.  In many cases, an estimate, regardless of its content and process by which it was created, is received implicitly as a pin point number with the accuracy of multiple decimal points.  This presents a problem for all parties involved.

I recently had a discussion with a gentleman who told me that prior to using our SLIM tool, the estimates in his organization were arrived at by casual hallway conversations, often started with, “how much do you think this will cost, and how long will it take?”  A typical response is, “hmmm, I’d say about 6 months and $500K.”  That innocent musing then becomes the information upon which business decisions are based, leadership bonuses may be won or lost, and the credibility of the dev team is on the line.

I’d strongly recommend adhering to some definitions when talking estimates.  These definitions will help mitigate potential misunderstandings around the agreement of what makes an estimate:

In my daily work life of supporting my clients, I’ve seen the IT project estimating function reside in many different areas of organizations.   Some groups have this function situated in a core group, such as an Estimating Center of Excellence, PMO or Delivery Excellence group.  These teams are responsible for either creating estimates for the organization after being fed some goals/assumptions, or vetting existing estimates born elsewhere to ensure they are in line with the business’ goals without compromising, as best they can, budget, schedule or quality.

Conversely, I’ve worked with organizations in which the project estimation function is supported by organizationally dispersed estimators who are busy with their other tasks.  Decentralized estimators typically serve a sole siloed division with not much interaction outside their team.  Like the bigger shops, these folks are funneled project estimates to be sanity checked and render their analysis with recommendations for adjusting the estimate if it seems out of line. 

Both scenarios above share a common thread – the estimates, regardless of their origin, have an impact on, and are influenced by, a wide breadth of people.  We at QSM call these people stakeholders, contributors, influencers etc.   They all have some kind of investment in the estimate that is important to them.  What if we could involve them in the estimating process?  With more communal agreement on inputs, there won’t be as much rework of the estimate later, when otherwise uninvolved people see their interests aren’t represented.  With everyone feeling they have a say in the estimate, smoother seas are at least more possible.

Managing vendors has become increasingly important in recent years.  In my account management role at QSM, I see both sides of the vendor management relationship.  The client wants a proven vendor that will partner with them in achieving their IT goals; and the vendor wants to win that business, employ their workers, and hopefully earn more work.  Unfortunately, that state of client/vendor Zen is not often achieved, usually due to legitimate (and sometimes not) misunderstandings on both sides.

On the client side, they are concerned with selecting a vendor with whom they are confident their tasks and deliverables will be achieved on time, within budget, and of the best possible quality.  After a round of RFI’s, then RFQ’s, then a final down select process, the vendor is chosen and work begins.  Often, at least in my experience, the overriding decision criteria comes down to cost, which makes sense, to a degree.  But in many cases, cheapest, I mean, least expensive bids often rule the day.  This kind of decision-making comes with its own set of risks; the most obvious is you get what you pay for and it’s often an ill-prepared vendor.

“No, we don’t have any historical project data collected” is the statement I hear with some frequency when speaking to organizations about their IT project estimating processes.  Ideally we use client history to calibrate and tune the project estimates we provide.  In my quest to spread the word about parametric estimating I often encounter this notion that organizations don’t believe they have historical data in a retrievable form.  In almost every case that I have been involved, it turned out that the historical data was present, just not in the form of a 1,000 rowed spreadsheet.  Often times the data is more available than the client is aware.

Our approach works at a macro level so we are seeking overall project metrics of cost, schedule, size, staffing and defects.  If the actual formal documentation of history is not available for these five core metrics, then it usually is available by leveraging various sources within the organization.  We have found it’s common to resurrect a project’s outcome by seeking feedback from the team that worked the project, however if that’s not possible due to attrition, re-org or other disrupting factors, we can usually find the project metrics through other means.  Those other means may be time and defect tracking tools, requirements analysis tools and accounting systems.  The data is almost always documented somewhere.   

Recently a friend of mine sent me a link to a YouTube video featuring Jeff Sutherland and Ken Schwaber, the founders of Scrum, discussing the latest update to the 2016 Scrum Guide.  The updates to the guide were comprised of survey feedback from the Scrum community, with the goal to understand what is important to them that should be included in the updated Scrum guide.  

The most compelling part of this discussion for anyone contemplating whether estimating belongs in Scrum happens at the 32:10 point.  Jeff and Ken are posed the question, “what is the difference between a forecast and an estimate?”   They answer with the idea that many people confuse the word estimate with commitment, which plagues any development effort regardless of method.  Subsequently Jeff and Ken changed the word “estimate” to “forecast” in the latest Scrum Guide to reflect the ever-changing growth and reduction in project scope, i.e., the estimate will show our best commitment and what we think is going to happen.  That can change once the project starts, BUT, we can measure that too through forecasting, essentially re-estimating throughout the project.  

Recently the correlation between seeking the best gasoline prices and software project overspending collided for me.  On one hand, IT projects will easily outspend their budgets, and on the other hand, in our private lives, we are doing all we can to save pennies in our personal budgets.

From time to time I have found myself pondering whether to drive the extra 5 miles out of my way in order to beat the system and enjoy the best gasoline price in my area.  I don’t believe I am alone in this quest as evidenced by the websites that publish the lowest gas prices within a certain radius.  Typically the competing stations have very small differences between their prices.  If I play my cards right, I may be able to save a total of $.50 - $.75 for a fill up.  In almost all other walks of life $.75 is not even worth a second thought.

Often I speak with IT project measurement folks about the permeation of agile into their project estimation processes.  While the Agile Manifesto recently celebrated its 15^th birthday, it’s just been the last several years that I’ve seen agile gain substantial momentum in becoming the official method many companies shepherd their projects.  Or has it…?

The graphic above shows that the use of agile over waterfall has yielded desirable results in terms of reaching business goals of delivering on time, within budget and with complete functionality.  Yet I still hear IT project measurement people voice concerns about fully adopting agile for their projects.

My observations, from talking to many people trying to estimate agile projects, are from a purely estimating perspective.  Before agile emerged, many development shops were building their projects via waterfall, ERP, RUP and other methods, all of which provided respective value.  Upon agile’s debut, it was but a whisper, among a few, as the next leading development methodology.  Over the years skeptical minds were slowly changed and adoption increased.  Today more organizations are using agile to develop their projects, but I have found a surprising number of estimators telling me in a hushed tone – they aren’t really developing in agile, despite the proclamation from on high.  They are using more a combination of agile and other development methods, yielding an agile hybrid, sometimes sheepishly referred to as “wagile” or “agilefall”.