Practical Software Estimation Measurement

Selecting the Right Software Estimation Tool for Your Business

Estimation Tool CHecklist

Organizations often come to us in the early stages of shopping for a software estimation tool and, oftentimes we find that they could be asking some additional questions. They often focus on the tool’s operating system, database structure, and architecture, when they could also be focusing on the quality of the data behind the tool. They also ask a lot of questions about inputting detailed information when really it would be in their best interest to focus on solid project-level information since detail-level inputs are often not available early in the planning lifecycle. Instead of focusing on the number of hours allotted to each individual person, it would be more beneficial to focus on how much work the overall team needs to finish.

In our 30+ years of experience in this industry, we've found that, no matter what tool an organization ultimately chooses, they need to be asking the right questions. Here is the criteria they should consider.

Tool Capability

As with any tool, it is important to match the tool with the job at hand. Using a screw driver to perform the task of a chisel will yield poor results. The same is true with trying to use a detailed planning tool in place of a software estimation tool. Make sure that you consider at what point in time formal estimates are required and how the resulting information is used to support negotiation and business decision making. Here are the main issues that should be taken into consideration when assessing an estimation tool.

  1. Product Maturity

A good product needs to be a mature product. Look for a solution that has a field use track record. What is the product operational lifetime in the marketplace?   How many companies and users are supported? Are the customers satisfied with the result? How accurate are the resulting estimates?

  1. Product Configurability

A good product needs to be able to be configured to your business and development environments. It needs to be able to speak your language.Can the product be configured to the way your company does business? Can the project lifecycle and milestones be customized? Can the Work Breakdown Structure be customized? Can the skills and defect categories be customized? Can sizing methods be customized to the engineering artifacts that are available at the time the estimate must be performed? 

  1. Ability to use customer historical data

The product should be able to accommodate historical data from your organization. Can the historical data be calibrated to reflect the application domains? Can the historical data be used to sanity check new estimates?  Can the historical data be stored in templates for estimating specific application domains?

  1. Adaptability to new technology & methodologies

The product should be flexible enough to accommodate new development methods and technologies. Does the product have configuration properties sufficient to model new ways of developing software? 

  1. Accessibility

A good solution should be easily accessible to the users, be it standalone PC, networked or cloud based. 

  1. Ease of Use

A good product should be easy to use.  Is the user interface well thought out? Does it allow efficient operation for the user? Will it significantly increase the efficiency of estimators? Are the outputs sufficient to support negotiation briefings and project planning?

  1. Ability to accept real world constraints

Does the tool allow solutions subject to project or organizational constraints (schedule, effort, staffing, and functionality)? 

  1. Efficiently generate and compare alternative scenarios

A good tool should allow for multiple alternatives to be compared. Does the tool allow for the capture, storage and comparison of alternative estimation scenarios? Is it easy to select and reload alternatives? 

  1. Ability to quantify and assess project risks

A good tool should be able to quantify the quality of the estimate relative to known uncertainties and risk. 

  1. Ability to compare estimates verses actuals

A good tool should provide the capability to monitor actual progress against the estimate. Can the tools provide comparisons of schedule, effort, staffing, functionality and defect actuals to baseline plans?

  1. Ability to provide robust estimation solutions

A good tool should provide as robust a solution as the problem demands. Can the tool estimate single and multiple releases? Do multiple release estimates incorporate dependency relationships? Can global updates be done on multi release estimates?

Supplier Capability

The supplier capability can be as important as the tool itself in a successful tool implementation.  You are buying more than a tool, you are selecting a partner to help you implement and deploy the solution.  These are important facts to consider about the supplier of your estimation tool.

  1. Business Stability

How long has the supplier been in business? Do they have a good reputation?  What does their customer profile look like? How stable is the employee base?

  1. Experience & Competence

How passionate are the people who support the product? How many years of domain specific experience do they have? What do their customers have to say about the competence of the supplier’s staff?

  1. Innovation Track Record

Is the supplier an innovator or a copier? Do they have a track record of bringing new ideas and products to market? Are they passionate about finding new and better ways to do things? Are they open to new ideas?

  1. Customer Support

What is the track record of helping clients over hurdles? Do they provide technical as well as professional help desk support? Are they responsive to your time line? What do current customers have to say about the customer support? Do they provide initial support sessions and mentoring?

  1. Deployment Support

To what extent is the supplier able to assist you in deployment, planning, and execution? Do they have teams that focus on deployment and change management?

  1. Training Support

Is there product training available? How is it made available - public, online, or in-house? How frequently is it offered? How much does it cost? How effective is the training and what do existing customers say about the training experience? Are there professional certification programs available?

  1. Consulting Support

Is additional consulting support available if required? What is the skill and track record of the services operation? 

  1. Tool Integration Support

Does the supplier have the capability to build or assist in the building of product integrations? Is there a published Application Programming Interface (API)?  Are there published examples of API integrations? 

Underlying Estimation Methodology

A tool is only as good as the underlying methodology. It is important to be able to explain the basis of the method to the people involved in the estimation process to help with credibility and adoption.  Here are some important issues you should consider.

  1. Sound Theoretical Basis

Is there a solid theoretical basis and framework that the tool is constructed around?  Is it intuitive and easy to explain?  

  1. Tool-Process Integration

Does the tool provide a capability to enhance process?  Is it flexible enough to adapt when processes change?

  1. Replicate  Reality

Can the tool replay projects that have already been completed to a satisfactory level of accuracy?

  1. Transparency to Stakeholders

Does the tool provide transparency into the estimation process?  Are the inputs easy to understand and explain?  Do the outputs relate to management issues like time, effort, cost, staffing, and reliability?

  1. Proven in your Industry

Is the method proven in your application domain and industry? 

  1. Accepts Artifacts/Data from Engineering Processes

Is the method flexible enough that it can be tailored to natural artifacts that result from normal engineering work on a project?

  1. Supports Closed Loop Process

Is the method a closed loop learning process? Does history feed estimates that are tracked to completion and is performance evaluated at the end of a project?

  1. Supports Effective Business Negotiation

Can the tool be used to support business negotiation at all levels of the organization? 

Price and Availability:

Price is always a major factor in purchasing transformative tools. It is important to evaluate the totality of the investment as well as the potential returns. You should be looking for the best value and you also want a great tool and a successful implementation. Here are a few pricing issues you should pay attention to.

  1. Price & Best Value

Look for the best value. It is a combination of price, support, stability and peace of mind.

  1. Implementation Flexibility

How flexible can the implementation be? Can one start with a pilot and scale up? 

  1. Deployment scope (Local - Global)

What is the scope of the likely deployment? Are there volume discounts for large purchases? Is there help desk and deployment support locally and globally?

There's a lot to consider when selecting an estimation tool. It's a monetary investment as well as a process investment. Ultimately, you want to know if it will help you accomplish the goals you set out to accomplish. Take the time to find a solution that will work best for you.  You will be glad that you did.

Blog Post Categories