QSM's recent webinar, Organizational Success; A Practical Guide to the Estimation Center of Excellence, presented by J.D. Ottenbreit, featured a lively Q&A session from our audience. Here are the highlights:
Q: You mentioned that data is critical to an Estimation Center of Excellence, what if we don’t have any good data yet?
A: The first thing I would say is that you need to chart a course to begin getting it. Your organization is most certainly creating data, but from an estimation standpoint you need to look for the very basic things like effort or hours spent on a project, whether that’s through your formal time recording system or basic timesheets that capture people’s contributions - ideally across the entire lifecycle and by project phase. If you’re not doing that, it doesn’t really matter if you’re estimating well or not; you’re missing a fundamental concept here. You should also look for quality and defect information in an honest way. And of course cost. Probably the one data point that gives people the most difficulty, at least from my experience, is scope or size. Of course size is an input into estimation; it’s not estimation itself and that’s a whole area we focus on as well. It’s critical. One of the most common errors that we see is when people just equate cost and schedule with the size of a project. From the perspective of our models, you need to at least quantify some sort of size of what it is you’re building and measure construction, not just consumption.
Of course, if you don’t have good data, QSM has one of the largest sources of completed project data. Many times clients that are interested in this journey will start with our data and then immediately begin collecting their own. The best practice is obviously to use your own data at some point, but sometimes that’s not always practical.
Q: How does QSM work with the client? Can QSM stand up the ECOE and run it or does the company always need to run it?
A: Each company is a little bit different. We have the capacity to help you stand it up and operate it, both on a short and a long term basis, but more often than not, this is knowledge that you want to transition into your own organization. However, we do have instances or cases where some clients are more than happy to outsource this function much like anything else. They have our teams permanently resident there. The clients have no interest in becoming professional estimators. They just want the outputs and the data to feed their decisions. So again, it depends on the level of investment, time, and interest that you might have.
Q: How much time do you estimate it takes to stand up an ECOE?
A: It’s going to depend on your organization and the number of people that you’re going to be impacting. If it is a broad endeavor at the enterprise level, it can anywhere from 6-18 months. If it’s training a small group of estimators with a small portfolio, it could probably be done in as short a period of time as 30-45 days. So it will vary. Some of our biggest clients who use our tools and methodologies across the board have taken years to fine-tune it, but that’s part of that continuous evolution. There’s a difference between standing up an Estimation Center of Excellence at a maturity one or two level and then have it operating like a finely-tuned machine at level five. So that’s a difficult question to answer. It’s one that we hear often, because in order to make a decision to invest in an ECoE, you need to know what’s your ROI and what’s the time period for that return. Sometimes it’s best to just start with a pilot in one particular area, look at the successes and then spread that across your organization. Very few executives are willing to sign off on this at the enterprise without at least some evidence that it’s going to work.
Q: You talked about how we all know the failure stories. What about the success stories? Do they exist in both the public and private sectors?
A: That’s a really good question. The success stories are often overlooked and sometimes the biggest ones are simply a recalibrated project or even a cancelled project. We have one success story within QSM, that I believe is public knowledge, to integrate many different systems in the federal departments. It was being pushed from the highest levels and they said they needed this done by a certain date and we were brought in to independently assess. We looked at the numbers and what they were trying to do based on the scope and size of the project. We used our models and our data and demonstrated that such a size project had never been accomplished before under those parameters. It was news that wasn’t received very well, but ultimately the data was very hard to argue with. It forced the decision-makers to go back and adjust their expectations around this particular initiative (and it was in the area of around $500-600 million).
The other success stores are simply making smarter investments internally and understanding how to use resources better. We had one client where we sat down with all of their PMs and asked them how they devised their estimates and they said, “we take a guess and we add 25% to that.” Well, 25% for every project in a large portfolio adds up pretty quickly, whether it is needed it or not. Think of the advantages of adding even half of those 25% contingency buffers together and funding a whole new project.
So the successes can be both large and small, but more than anything, it gives you more visibility into the decisions you’re making in your organization. It can be a very rewarding and gratifying thing for professionals in this field to have an impact on their company.
Q: Are Estimation Centers of Excellence relatively new? Have any large private corporations engaged QSM to stand up an ECOE and if so, why?
A: The term Estimation Center of Excellence is rather generic. We started using it several years ago and it caught on, but the notion of the center of excellence is not new. We’ve been promoting better use of tools and methodologies for decades. The Center of Excellence concept itself brings it all together under one umbrella because it’s all about that continuous improvement. And yes we do have some very large commercial clients in addition to some government clients that have adopted this approach. They adopt ECOEs because they’re struggling with some things. They’re not sure if they’re getting a return on their investments. We have a whole solution designed around vendor management that’s been getting a lot of interest in the market. Big companies that are outsourcing large development contracts feel like they’re getting bullied. They don’t know whether they should be getting charged $5 million or $100 million, but they don’t have any data to negotiate with the vendors and so we help them do that in an objective and independent way based on our approach to using quantitative methods to better manage contracts. That’s just a single instance of why you would want to do this – do you know if you’re getting taken advantage of simply because you don’t have the right information? We’ve also been involved in litigation in the form of expert witness testimony where lawsuits are brought against companies for going so far past their contracted amount and not delivering on a contract. So we determine if this was even possible to begin with or quantify the scope of the mistake. So there are some private organizations that might use it for those purposes.
Q: What do you think are some of the most important offerings or areas an ECOE should focus on?
A: One of the most obvious things is that an ECoE is a place to put all the templates, guidance, training, and instruction. You need to create that community to begin with, and we’ve seen that happen very effectively once it catches on. We often say that the estimation needs to occur closest to the projects. What you want to do is create an environment that fosters innovation that allows your project managers and divisions to operate creatively in their own domains, but you still give them the structures, governance, and consistency to feed the organization the data that it needs to make better decisions. So, if anything, the Center of Excellence can coalesce around those concepts and provide the forum for training, certification, and communication - both up to the executives and into the rank and file. Without that center of gravity, then you might have pockets of good work being done but there’s no spotlight on it that could help justify funding, highlight all the benefits and capture the praise the goes with the success.