New! Check out the QSM Software Almanac: 2019 Edition to see the latest research from this database.
The QSM database is the cornerstone of our business. We use validated metrics collected from over 10,000 completed software projects to keep our products current with the latest tools and methods, to support our benchmarking business, to inform our customers as they move into new areas, and to develop better predictive algorithms.
Data Sources
Since 1978, QSM has collected completed project data from licensed SLIM Suite users and trained QSM consulting staff. Consulting data is also collected by permission during productivity assessment, benchmark, software estimation, project audit, and cost-to-complete engagements. Many projects in our database are subject to non-disclosure agreements; but regardless of whether formal agreements are in place, it is our policy to guard the confidentiality and identity of all data contributors. To preclude identification of individual projects/companies or disclosure of sensitive business information, we release industry data in summary form only.
In 1994, QSM began collecting project data continuously, updating the database every 2-3 years. Over the last 5 years, we have added an average of 200 validated projects each year.
Data Quality
Only projects rated Medium or High confidence are used in QSM’s industry trend lines and research. Before being added to the database, incoming projects are carefully screened. On average, we reject about one third of the projects screened per update.
Data Metrics
Our basic metric set focuses on size, time, effort, and defects for the Feasibility, Requirements/Design, Code/Test, and Maintenance phases. These core measurements are supplemented by nearly 300 other quantitative and qualitative metrics. Approximately 98% of our projects have time and effort data for the Code and Test phase and 70% have time/effort data for both the R&D and C&T phases.
Industry Data
QSM data is stratified into 9 major application domains (Avionics, IT, Command & Control, Microcode, Process Control, Real Time, Scientific, System Software, and Telecom) and 45 sub-domains. Software projects predominate, but we have a growing number of hardware and infrastructure (non-software call center) projects as well.
Data contributors include DoD; civilian commercial firms; and national, state, and local government entities. In addition to domain complexity bins, our data is also broken out by major industry and industry sector. Major industries include the financial sector, banking, insurance, manufacturing, telecommunications, systems integration, medical, aerospace, utilities, defense, and government.
Methodology Data
The QSM database includes a variety of lifecycle and development methodologies (Incremental, Agile, RUP, Spiral, Waterfall, Object Oriented) and standards (CMM/CMMI, DoD, ISO).
Language Data
Over 700 languages are represented with most projects recording multiple languages. Common primary languages are JAVA, COBOL, C, C++, C#, VISUAL BASIC, .NET, IEF / COOLGEN, PL/1, ABAP, SQL, ORACLE, POWERBUILDER, SABRETALK, JAVA SCRIPT, DATASTAGE, HTML. Frequently used secondary languages include JCL, SQL, JAVA, COBOL, ASSEMBLER, C++, HTML, VISUAL BASIC, XML, ASP.NET, and JSP.
Country Data
QSM has collected and analyzed software projects from North America, Europe, Asia, Australia, and Africa. About 50% of our data is from the U.S. Another 35-40% is from India, Japan, the Netherlands, the United Kingdom, Germany, France, and other major European countries.
This SLIM-Metrics chart shows Construction & Test effort for completed IT, Engineering, and Real Time systems as System Size increases. QSM stratifies project data into homogenous subsets to reduce variation and study the behavioral characteristics of different software application domains.
QSM Industry Trend Lines
QSM industry trend lines are available for nine, high level application domains, five application subgroups, and three application super groups. The nine application domains are:
| Business | Command & Control | Scientific |
| System Software | Telecommunications | Process Control |
| Avionics | Microcode/Firmware | Real-time Embedded |
Stratifying the data by application type reduces the variability at each size range and allows for more accurate curve fitting. One application domain, Business IT projects, has been further stratified into several sub-groupings:
| Business Agile | Package Implementation |
| Government | Web Systems |
| Business Financial |
Three application super groups are also available to benchmark projects of mixed or unknown application domains:
| Real Time Group | Engineering Group |
| All Systems |
There are several ways QSM clients can access the QSM database:
- Our SLIM Tools Contain Current Industry Performance Trends: QSM provides up to date trends for 17 application types with QSM software project estimation and benchmarking tools.
- We Answer Your Basic Questions: Our support team is happy to answer basic questions about the QSM database. Let us do the research to answer your estimation and benchmarking questions! We can provide graphs and summaries that allow you to compare your projects against both industry trend lines and actual projects that are similar in size, application type, and complexity. Basic questions usually involve research that we can accomplish in less than 4 hours. Note: more extensive research is available through our consulting group.
- We Provide Benchmarking Services: QSM’s benchmarking service can help you assess your current levels of productivity and quality, identify exceptional or underperforming projects, analyze the root causes of poor performance and provide a roadmap for improvement, and build a business case for process improvement initiatives. Our benchmark service will position your projects against relevant industry data and measure your variation from the norm.
- We Provide Client-Directed Research: QSM can be engaged to perform research and answer specific questions about performance, cause and effect relationships, or the impact of various tools or practices on software development projects. Current research from our database analysis is also available for download from our website and blog.