Articles & Papers
In the interest of supporting the software development industry, the following resources are available free of charge. Click a category to jump to the corresponding section:
Articles and White Papers (PDF and HTML)
Forecasting from Defect Signals by Paul Below. On large software development and acquisition programs, testing phases typically extend over many months. It is important to forecast the quality of the software at that future time when the schedule calls for testing to be complete. In this article, originally published in CrossTalk, Paul Below shows how Walter Shewhart’s Control Charts can be applied to this purpose, in order to detect a signal that indicates a significant change in the state of the software. September, 2014.
Set the Stage for Software Project Success by Don Beckett. Management decisions made before a software project is underway are a significant factor in determining whether it succeeds or fails. In a recent article for Projects at Work, QSM's Don Beckett identifies seven principles, based on comprehensive studies, that leaders must support and uphold to help create an environment in which projects can succeed. Ignoring them practically guarantees failure. May, 2014.
Software Estimation: How Misperceptions Mean We Almost Always Get It Wrong by Carol Dekkers. In this highly-discussed article for Dr. Dobb's, QSM's Carol Dekkers asks a tough question: why are we so woefully poor at estimating software projects? It's a tough pill to swallow considering software developers are among the smartest people on the planet, often boasting advanced degrees in mathematics, engineering, or computer science. Yet study upon study cites that less than one-third of projects are delivered on time or on budget. The problem of software project estimation is not straightforward. To get the heart of the issue, Carol Dekkers takes us through the five top misperceptions about software estimating, and what we can do to address them. April, 2014.
Big Agile: Enterprise Savior or Oxymoron? by Larry Putnam, Jr. We know agile works well for small teams and small projects, but monster enterprise projects often require greater capabilities than a small team can provide. So why not scale up agile teams to maintain the cost and efficiency benefits of the agile process while accessing the necessary manpower to pursue complex global projects? On the surface, it makes sense, but what if agile only works when teams and projects stay relatively small? That's the question most CIOs want answered before investing scarce time, energy, or resources chasing the big agile paradigm. In this article originally published on Agile Connection, QSM's Larry Putnam, Jr. turns to cold hard data from completed projects in the QSM database to determine whether big agile is "enterprise savior or oxymoron." March, 2014.
Project Clairvoyance by Larry Putnam, Jr. Can advances in data-driven estimation turn software project failure into a distant memory? Well, if learning from experience is the key to success, imagine what you could do with real-time access to three decades of research, thousands of projects and more than 600 industry trends. In this article, originally published on Projects at Work, QSM's Larry Putnam, Jr. identifies key benefits of employing estimation best practices for project success. January, 2014.
Constant Velocity Is a Myth by Andy Berner. Is your agile team’s velocity constant from sprint to sprint? No? That’s not a surprise. Many teams assume that their velocity will be constant. In this article, the third in a series recently published on Projectmanagement.com, QSM's Andy Berner explains why that’s not the right expectation--and how that affects how you use this metric. December, 2013.
Ready, Set, Go...and Ready Again: Planning to Groom the Backlog by Andy Berner. In an agile project, the backlog (the prioritized set of requirements) is the main input to iteration planning. For an agile project to progress smoothly, the backlog must be groomed and ready for each sprint. That work must be included in your project plan. In this article, the second in a series recently published on Projectmanagement.com, QSM's Andy Berner gives you five points to consider when planning that work.
An Analysis of Function Point Trends by Don Beckett. Function point analysis has played an important role in software measurement and analysis for 30 years. This study looks at the QSM software project database and examines a set of validated projects counted in function points that have completed since the year 2000 to see what they tell about productivity, schedule, and staffing. We are fortunate to have several thousand projects in this sample to work with as this allows us to parse the data many different ways and still have enough projects to be statistically significant. For this study only unadjusted function points were used. August, 2013.
Is It Bigger than a Breadbox? Getting Started with Release Estimation by Andy Berner. It’s becoming clear to organizations adopting Agile methods that one still needs to estimate how long a project or a release of a product will take. It won’t suffice for businesses to simply take guesses or accept unreasonable constraints. We must be able to derive credible estimates, based on a history of similar projects. But how can we estimate a project in advance, while still maintaining the ability to manage the backlog in an Agile manner? In this article, Andy Berner answers that question, compares release-level estimation to the techniques used for iteration estimation, and gives some pointers on getting started with release estimation in an Agile environment. Originally published on Projectmanagement.com, July, 2013.
Does Agile Scale? by Larry Putnam, Jr. What happens to Agile projects when they’re forced to scale to the size of major government enterprise initiatives? In this article, originally published in the May-June 2013 issue of Modern Government, Larry Putnam, Jr. looks at 93 Agile projects completed between 2002-2012 to see how these projects fared as their sizes increased. July, 2013.
Data-Driven Estimation, Management Lead to High Quality by Kate Armel. Quality assurance comprises a growing share of software development costs. To improve reliability, projects should focus as much effort on upfront planning an estimation as they do on remedial testing and defect removal. Industry data show that simple changes like using smaller teams, capturing average deviations between estimates and actuals and using this information as an explicit input to future estimates, and tuning estimates to an organization's historical performance result in lower defect creation rates. Access to accurate historical data helps projects counter unrealistic expectations and negotiate plans that support quality instead of undermining it. Originally published in Software Quality Professional, March, 2013.
Predictable Change: Flexing the Five Core Levers of Software Development by Andy Berner. In this white paper, Andy Berner introduces the key metrics used to predict what it takes to do a new project and some of the issues you’ll encounter when moving from Agile iteration planning to planning new projects and releases. September, 2012.
The Difference Engine by Phillip Armour. Originally published in Communications of the ACM, "The Difference Engine" focuses on building teams of differently skilled people. The article is partly based on University of Michigan Professor of Complex Systems, Scott Page’s book, The Difference, which shows the power of cognitive diversity in building systems and solving problems. January, 2012.
Technology Can Only Do So Much by Kate Armel. Kate Armel examines the applicability of Fred Brooks' age-old maxim - “Adding manpower to a late software project makes it later.” - in today’s IT development environment. Using data from the QSM Database and research performed by QSM’s Don Beckett, Kate analyzes projects sized from 50K to 500K ESLOC to determine whether project size has any effect on the point at which adding more people causes a project to come unglued. Is Brooks’ statement still universally true? The results might surprise you… March, 2011.
Data Mining for Process Improvement By Paul Below. What do you do if you want to improve a process and have 100 candidate predictor variables? How do you decide where to direct causal analysis effort? Similarly, what if you want to create an estimating model and you have so many factors you do not know where to start? Data mining techniques can be used to filter many variables down to a vital few to attack first or to build estimating models to predict important outcomes. Originally published in CrossTalk Magazine, January 2011.
Making The Right Choices For Healthy Software - By David Rubenstein. SD Times looks at the latest QSM research into best- and worst-of-class projects to see how successful software development managers estimate, plan, and manage projects. January 2006.
How QSM Products Support PMI® Knowledge Areas A convenient matrix showing how QSM tools support different PMI® Key Knowledge Areas.
SLIM and Christopher Columbus - By Ernst Van Waning.The commercial development of software is not a process that can be done 'by the eye': the long accepted software crisis is now seen as a chronic situation facing the IT industry. Developing software is hard enough but managing software development projects is even harder to do 'by eye'. Tools are needed to show your development process as it really is: in project execution, in plans and bids and in comparison to other projects in the market. Easy to say, but what does it actually mean? August 2005. PDF version
"Ensuring Delivery of Highly Reliable, Complex Software Releases" - By Jim Greene.A case study from major telecom supplier demonstrates how to estimate, plan, and manage distributed subsystems that are developed separately, then integrated to form a release. Examines the challenges of delivering a distributed, high-reliability system. October 2003.
"Telecom Software Benchmarks: 10 Years Apart" - By Jim Greene.Benchmarks taken 10 years apart measure the benefits of process improvement initiatives and the impact of increased time-to-market pressures on a major telecom supplier. September 2003.
"How Much Software Can Be Built In A Year " - By Doug Putnam.In today’s world, customers expect to see software projects delivered in increasingly shorter cycles. The maximum time most IT customers seem willing to wait is about one year. In this fast-paced environment, the question becomes: just how much software can a typical IT shop create in 12 months?March 2002.
"What I Did Last Summer: A Software Development Benchmarking Case Study " - By James T. Heires.This article describes a vendor-supported benchmarking study of an applications development department. The study established a quantitative performance baseline of the organization and compared it to industry trends. October 2001.
"QSM Software Research Bulletin" - By Doug Putnam.Results of a long-term productivity study covering the period from 1982-1997 in condensed format. November 2001.
"Software Quality Assurance (SQA) of Management Processes Using the SEI Core Measures" - By Jim Greene.Using the right metrics for the three key management processes - benchmarking, estimation and risk analysis, and project tracking and control - to deliver a quality product. September 19, 2001.
"Build in Quality" - By Lawrence H. Putnam Sr. and Ware MyersQuality is the positive side of quality-defect continuum. The use of metrics is fundamental to allow sufficient time and effort to build a quality product. April 2001.
"Build it Faster!" - By Lawrence H. Putnam Sr. and Ware Myers How can we build software faster? Specifically, how can metrics help reduce time-to-market? April 2001.
"Productivity Statistics Buck 15-Year Trend" - By Douglas Putnam. Examines the effects of new technologies such as ERP, object-oriented development, and Internet/e-commerce on long-term productivity trends from the QSM database. July 2001.
"Software by the Numbers: an Aerial View of the Software Metrics Landscape" - By Michael Mah and Lawrence H. Putnam, Sr. As we muse on what 'new metrics' apply in the ever-changing world of software, we should also be asking ourselves the more basic question, 'What will we do with metrics?' . Many organizations don't have the basics down before they pile on measure after measure. This article originally appeared in Vol. 10, no. 11 of AMERICAN PROGRAMMER. 1998.
"Linking the QSM Productivity Index to the SEI Maturity Levels" - By Lawrence H. Putnam, Sr. This paper explores the correlation between QSM's Process Productivity Index (PI) and SEI CMM level. July 2000.
"Purchasing Software Intensive Systems Using Quality Targets" - By Jim Greene. DOD process for tracking reliability against quality targets. November 2000.
"16 Critical Software Practices For Performance-Based Management" - The Airlie Software Council was formed in late 1994 under the coordination of Dr. Norm Brown of the DoD Software Program Managers Network. Its mission is "to identify best software practices in Government and Industry, and to cause these best practices to be implemented in DoD software management, development and maintenance".
"What We Have Learned" - By Lawrence H. Putnam and Ware Myers. Lessons learned from studying the SEI core metrics and process productivity (PI). Published in Crosstalk, April 2000.
"Without Software, No Megatrends" - By Lawrence H. Putnam and Ware Myers. Improving software productivity is the key to supporting the megatrends of the future.
"Key Things We Have Learned" - By Lawrence H. Putnam and Ware Myers. A retrospective view on how Larry Putnam's seminal thoughts (and one new observation) on software development have held up over the past 25 years. February 2000.
"The Princess and the Pea" - By Lawrence H. Putnam and Ware Myers.
"A Throughput Measurement Procedure Using SLIM" - By Lawrence H. Putnam. How to calculate and demonstrate the value of staffing and process improvement strategies using SLIM-Estimate.
"A Process for Costing Requirements Changes" - By Mike Ross.
"Analysis of On-Board Spacecraft Software Development" - By Jim Greene.
"Avoiding the Premature Delivery of Software" - By Jim Greene.
"Estimating Software Size and Uncertainty" - By Jim Greene.
"Estimating When You Are Moving to New Technologies" - By Doug Putnam.
"First, Get the Front End Right" - By Lawrence H. Putnam, Michael Mah, and Ware Myers.
"Home Runs in Management Science" - By Doug Putnam.
"Independent Research on Software Reuse" - By Mike Mah and Ira Grossman.
"Larry Putnam's Career in Software Estimation" - By Larry Putnam, Sr.
"Managing Major Distributed Software Development" - By Jim Greene.
"Measures For Software Acquisition" - By Jim Greene.
"Product Review -- SLIM-Metrics®: A Powerful New Repository and Analysis Tool" - By James T. Heires, Rockwell Collins, Inc. See also ADT Magazine, June 1998 issue.
"QSM Reliability Model." - By Doug Putnam.
"Size Does Matter: Continuous Size Estimating and Tracking" - By Mike Ross.
"Sizing and Controlling Incremental Development" - By Jim Greene.
"The (Almost) Perfect Software Project Using the SEI Core Measures" - By Jim Greene.
"Haste Makes Waste When You Over-Staff to Achieve Schedule Compression" - By Doug Putnam.
"Simple Project Tracking Approach Puts You in Control of Your Project" - By Doug Putnam.
"You Need the Right Map to Know Where You're Going With Process Improvement" - By Doug Putnam.
"Team Size Can Be the Key to a Successful Project" - By Doug Putnam.
The Familiar Metric Management Series was written by Larry Putnam, Sr. and Ware Myers for Cutter Consortium's IT Metrics Strategies.
(32) Effort, Development Time, and Defects Interact
(31) Get Yourself a Little Whip!
(30) Multiple Uses of Software Metrics
(29) Year 2000 Turns the Mythical Man-Month on its Head
(28) Small is Beautiful-Once Again
(27) QSM Database Shows Drop in Productivity
(26) Management Vision Precedes Metrics
(25) "Let Me Count the Ways"
(24) Experience the Wisdom That Goes With Knowing Where You've Been
(23) Know What You Have to Do
(22) Leonardo da Vinci Knew Feasibility
(21) Come to dinner, Henry!
(20) From Work to the Bit-Stream
(19) "The software cost estimation problem is solved!"
(18) What Will the Year 2000 Fix Cost Me?
(17) Productivity of the Reuse Organization
(16) Software Reuse
(15) The Future of Metrics in Software Development
(14) Produce More Systems
(13) The Effort-Time Tradeoff: It's In The Data
(12) The Power of the Tradeoff
(11) Negotiate from a Fact-Based Position
(10) Evaluating a Technology
(7) Executive Backing
(6) Software Adds Business Value
(5) Time Boxing
(2) Familiar Metric Management +
(1) Familiar Metric Management
QSM periodically publishes a newsletter aimed at keeping our current and prospective customers informed about new developments and issues relevant to software project management. Back issues of our newsletter are available as PDF files for download, and can be viewed with the Adobe Acrobat® Reader. Click on the issue number to download the desired newsletter.
February 2012 Newsletter
November 2011 Newsletter
April 2011 Newsletter
February 2011 Newsletter
April 2010 Newsletter
January 2010 Newsletter
October 2009 Newsletter
QSM Launches New Website and Blog
March 2008 QSM News Alert
January 2008 QSM News Alert
October 2007 QSM News Alert
May 2007 QSM News Alert
September 2006 QSM News Alert
July 2006 QSM News Alert
April 2006 QSM News Alert
February 2006 QSM News Alert
December 2005 QSM News Alert
August 2005 QSM News Alert
November 2003 QSM News Alert
July 2003 QSM News Alert
QSM Perspectives Volume 20 Number 2 (Fall 1997)
QSM Perspectives Volume 20 Number 1 (Spring 1997)
QSM Perspectives Volume 19 Number 2 (Winter 1996)
PMI® is a registered mark of Project Management Institute, Inc.