Keith Ciocco's blog

Keith Ciocco's blog

How Can Organizations Optimize Costs in the IT Budgeting Process?

It’s that time of year again for many C-level executives: time to figure out the IT budget for next year. This is to bring the business side of the organization to the table with the technical side to forecast how much IT is going to spend. It can be a complicated process, but there are ways to make it easier and more accurate; and there are ways to save a lot of time and money. The challenges often relate to short planning time frames, minimal information available to generate accurate forecasts, political agendas within the organization, and, unfortunately, only a small number of estimation methods in place. But there are tools and processes available to help face these challenges. Here are the basic steps that we recommend for cost optimization in the budgeting process.

Start by analyzing the historical data that is available. The process can be streamlined by focusing on the core metrics within the organization. This data can include release level size, effort, staff, and duration information. Historical data showing typical effort by role by month spending is also valuable to leverage. Ideally, this type of data should be captured on 8-15 projects.

The next step is to pull together scope level sizing data on projects that are being considered for the new year. This information can include epics, themes, user stories, business requirements, or use cases, to name a few. The goal here is to get as close as possible to determining how much work needs to be done on each release in the pipeline. Once there is a large enough sample of data, then release level estimates can be created for the coming year. There are tools available to help streamline this process and the best ones allow for risk mitigation and sanity checking with historical data.  

Blog Post Categories 
IT Budgeting Estimation

Agile On-Time, But Is It Reliable?

With agile projects, we hear a lot about the planning benefits of having a fixed number of people with a fixed number of sprints.  All great stuff when it comes to finishing on time and within budget. But one of the things we also need to focus on is the quality of the software.  We often hear stories about functionality getting put on hold because of reliability goals not being met.

There are some agile estimation models available to help with this, and they can provide this information at the release level, before the project starts or during those early sprints. They provide this information by leveraging historical data along with time-tested forecasting models that are built to support agile projects. 

In the first view, you can see the estimate for the number of defects remaining. This is a big picture view of the overall release. Product managers and anyone concerned with client satisfaction can use these models to predict when the software will be reliable enough for delivery to the customer.

MTTD over Time

In the second view, you can see the total MTTD (Mean Time to Defect) and the MTTD by severity level. The MTTD is the amount of time that elapses between discovered defects. Each chart shows the months progressing on the horizontal axis and the MTTD (in days) improve over time on the vertical axis. 

Mean Time to Defect

Blog Post Categories 
Agile Quality Estimation Software Reliability

Selecting the Right Software Estimation Tool for Your Business

Estimation Tool CHecklist

Organizations often come to us in the early stages of shopping for a software estimation tool and, oftentimes we find that they could be asking some additional questions. They often focus on the tool’s operating system, database structure, and architecture, when they could also be focusing on the quality of the data behind the tool. They also ask a lot of questions about inputting detailed information when really it would be in their best interest to focus on solid project-level information since detail-level inputs are often not available early in the planning lifecycle. Instead of focusing on the number of hours allotted to each individual person, it would be more beneficial to focus on how much work the overall team needs to finish.

In our 30+ years of experience in this industry, we've found that, no matter what tool an organization ultimately chooses, they need to be asking the right questions. Here is the criteria they should consider.

Tool Capability

As with any tool, it is important to match the tool with the job at hand. Using a screw driver to perform the task of a chisel will yield poor results. The same is true with trying to use a detailed planning tool in place of a software estimation tool. Make sure that you consider at what point in time formal estimates are required and how the resulting information is used to support negotiation and business decision making. Here are the main issues that should be taken into consideration when assessing an estimation tool.

Blog Post Categories 

Can We Increase IT Productivity by Leveraging Predictive Analytics?

Often within technology organizations there is a general belief that increasing staff increases the amount of production. But what if there were better options? Wouldn’t it be great to see some additional management options using predictive analytics? This type of analysis could save organizations millions of dollars by showing how to hit their goals by just planning more effectively.

Where do you start? First, we recommend centralizing your project data so your information can be easily accessed. These projects can be completed, in-progress, or getting ready to start. The best way to do this is with a tool that lets you store the data and that also lets you generate the forecasts, all in one convenient place.

Software Project Portfolio

The next step would be to run built-in forecasting models to see if you can complete the required amount of work with the existing number of resources.  These models also provide other options to consider, like adjusting the number of resources on a software release or extending a project schedule to save money. The best models are empirically-based and time-tested. To generate the analysis, you need to enter some basic project level goals and the models then leverage historical data to forecast a reliable duration, effort, cost, and staffing assessment for each release.

Software Project Data

Blog Post Categories 
Productivity Portfolio Analysis

How Can We Leverage Summary Level Analytics to Support Enterprise Planning?

What if you could leverage summary level cost, duration, and productivity data to support estimates for future projects, at the release and enterprise level? C-level executives, development managers, and project stakeholders are all involved at some level in project planning. They want quick access to information on a regular basis and they want web-based solutions to make it happen. So how does it all work? There are web-based analytics tools that allow you to create a centralized database for all of your projects. These tools store the data, leverage it to generate project and portfolio estimates, and then provide a communication vehicle throughout the organization to ensure that everyone involved is on the same page. It all starts with having the data in one place. 

Software Project Database

Once you have all of your project data in one place, then you can focus on analyzing the completed projects. You can compare them against industry trends and leverage a 5-star report to show how they rate on performance in the industry. The initial measures to focus on would be size, duration, effort, reliability, and productivity. A project's productivity will be calculated automatically once you have entered the size, duration and effort. We call this measure a Productivity Index. This measure can be compared to industry and used as a benchmark to measure process improvements over time.  These numbers give you a quantitative picture of your current project environment.  

Software Project Closeout

5 Star Report

What do Sports and Agile Software Productivity Have in Common?

I am a big sports fan and since I work for a software metrics company, I started thinking about the similarities in productivity measurement in both industries. From draft picks to game planning decisions, managers in sports measure their team’s productivity to help them make better decisions in the future. Software executives and product owners do the same thing; they measure productivity in order to make better planning decisions regarding upcoming projects.

To measure productivity in baseball, we look at measures like batting average, on-base percentage, slugging percentage, and earned run average. When measuring the productivity of agile software projects, we often look at the velocity, which takes into account the number of user stories completed in each sprint. This type of historical data helps us plan effectively at the detailed level.

As part of our work at QSM, we are often asked to provide plans at the release and portfolio level. To provide these plans reliably, we use a macro level, empirically-based productivity measure that encapsulates a number of project-related factors. This measure is called the Productivity Index, an integral part of the Putnam Model.  Also known as the SLIM Model, the Putnam Model was invented by Larry Putnam Sr. almost 40 years ago and is having a big impact on software measurement more than ever today.

Once we know the total number of user stories (or any size measure), the release level effort, and the duration, we can calculate the Productivity Index of a project. The Productivity Index also takes into account the project environment including: the experience level of the team, the complexity of the software, and the quality of the tools and methods being used on the project.

Blog Post Categories 
Productivity Agile

QSM attends Gartner Conference with Big Focus on IT Budgeting and Vendor Management Initiatives

QSM Booth at Gartner Symposium

The QSM team thoroughly enjoyed our time at the Gartner Symposium/ITXPO in Orlando a couple of weeks ago. We spent most of our time at the QSM exhibit, networking sessions, and at various presentation sessions.

We met many C-Level executives with process improvement on their mind. Our main focus was to learn about their specific needs in the IT budgeting and vendor management areas. We conducted question and answer sessions and provided real world examples on how to use business and software analytics to manage the complexities of IT budgeting, taking into account in-house projects as well as the project durations and costs proposed by their vendors.

Many of the senior executives we spoke with also had a need to integrate cost, duration and resource information with their project portfolio management solutions. The big focus was leveraging demand management to improve capacity planning. We provided demonstrations showing the SLIM-PPM Integration Framework which provides the release level cost, effort, resources and schedules that PPM solutions need validated to reduce risk in the enterprise portfolio.

Blog Post Categories 
QSM News IT Budgeting

Is There a Better Way to Do Agile Planning?

Plan Agile Projects BetterThere are so many questions around agile planning, one of the biggest being: do we need an estimate? Project managers and scrum masters will spend months developing a system either for internal use or for their clients, yet some of them say that estimates are not needed. Some recommend starting the project without an estimate. They say they will see how the first few weeks go before they generate an estimate. Others say not to worry about an estimate at all; they are a waste of time.

The problem with those recommendations is that there are business decisions that need to be made regarding whether or not to even start the project. Reliable estimates for cost and duration are needed to make these decisions. Also, for the projects that do move forward, there is usually limited information available early in the lifecycle, not enough to provide a detailed plan. Product owners need to see the big picture before a reliable detailed plan is generated.

There is also the IT manager that needs to figure out how they will allocate their resources. There is the vendor manager that needs to evaluate multiple bids for a large software system that could cost the company millions of dollars. There is the proposal manager that needs to write a proposal that must be cost competitive to win business. And, there is an annual budget at stake and the CIO needs to know how much money their development organization is going to spend over the next 12 months. You can’t support these decisions in the best way possible without reliable release level estimates.

Blog Post Categories 
Agile Estimation

Can We Take the Tracking of Agile Projects to the Next Level?

Before an agile project starts, many product owners will run an early release estimate. Once the activities get started, managers or scrum masters begin to track the progress. When they track, they usually include the person hours of effort and the number of user stories within each sprint.  There are a number of agile tracking tools and methods in the marketplace for these tasks. 

But wouldn’t it be great if the tracking and estimation process could be combined, using the actual tracked effort and user stories to run new and improved ongoing estimates at the release level? At QSM, we have applied this process to hundreds of software projects. This type of adaptive forecasting can help save time and effort by showing when a software release is headed down the wrong path. It can also help organizations avoid signing up to inflated resource planning numbers that cause many companies to waste millions of dollars at the release and enterprise levels.

In the SLIM-Control charts below, we see the blue plans versus the red actuals and the new forecasts in white. We are capturing the total effort spent and the actual work delivered each week, then using that information to generate mathematical models that produce new empirically based forecasts at the release level.

Agile Project Tracking

Blog Post Categories 
Agile SLIM-Control

How Can We Make Annual IT Budgeting Easier?

One of the things I hear from many c-level managers is how difficult it is and how long it takes to generate reliable resource plans at the enterprise level. Many organizations take months to generate their annual budgets and often times the negotiated budgets end up being unrealistic. To fix this problem we need to combine good capacity planning with good demand management. There are a number of project portfolio management tools to help with the capacity planning. The problem is the numbers will be off if we don’t get the demand management part right.

There are world class demand management tools available that can be used by business decision makers. These tools allow us to come up with empirically based, reliable project and enterprise level resource plans. In the SLIM-Estimate view below you can see the forecasted effort by role by month.

IT Budget Planning

The view below shows the plan being sanity checked with industry data so we can better negotiate our budgets with confidence.

IT Budget Planning

The even bigger news is that we can automatically feed our empirically based demand numbers into our PPM tools, making the job of capacity planning much easier and more reliable. In the view below you can see two separate plans, represented in side by side columns. The original PPM plan was updated automatically from an empirically based and sanity checked plan from SLIM-Estimate.