Two common questions we receive are “how quickly can we get SLIM deployed within our organization?” and “how accurate will our estimates be?” On the accuracy question, our answer is usually that “it depends”. Like most things, accuracy and success in estimating are directly correlated to the effort put into the task. If you buy a commercial estimating tool and try to use it “out of the box” with no tailoring and calibration, accuracy usually suffers. One of the most efficient ways to achieve both of these goals is to use your own historical data. This was an excellent example of how one of our global systems integration clients was able to get almost immediate value by calibrating SLIM-Estimate to their historical data.

This major integrator wanted to improve estimation at one of their accounts, a large multinational company.  In addition they wanted to use function points as their sizing metric. After evaluating a number of estimation options, including custom built and other internally developed solutions, they decided to use QSM’s SLIM-Estimate, which was being deployed globally across the rest of the organization. They decided to start with a pilot implementation. The estimation pilot lasted a total of 5 months with a staff of 2 people. After the fact they looked at the effort expended on their pilot activities and found that 60% of the total effort went into their calibration activities. The calibration  process included assembling a historical data sample of over 50 projects. As part of their function point deployment, they empirically determined FP/ESLOC gearing factors.

In recent years, we have seen our client base become increasingly diverse, expressing the need for our estimation, tracking, and benchmarking tools for design processes outside of just software. While clients have customized SLIM to other design disciplines in the past, our goal with SLIM 8.0 was to increase configurability within our tools so our users can model any type of system quickly and easily. Now users can forecast and benchmark Agile, infrastructure, offshore/multi-shore, ERP/package implementation projects, and more.  In addition to updated trendlines from 10,000 completed software and systems projects, SLIM comes pre-packaged with trend groups, such as Agile, ERP, Financial, Web, and Government.

Read the full press release with detailed product upgrades here.

Last week we looked at IT software productivity trends for 1000 completed IT systems and noted that average productivity has declined over the last 15 years.

The post sparked some interesting responses. Two readers wanted to know whether productivity actually increases over time for projects in the same size range? If so, this would be an illustration of Simpson's Paradox: a counterintuitive phenomenon we've seen from time to time in our own research. Simply put, sometimes the direction of a trend is reversed when the sample is broken into categories.

To answer their question, I used our SLIM-Metrics tool to stratify the sample into four size bins:

Under 5000 Effective (new + modified) SLOC
5000- <10000 Effective (new + modified) SLOC
10000-<20000 Effective (new + modified) SLOC
20000-<30000 Effective (new + modified) SLOC

These 4 size bins span a little over two thirds of the data. As a sanity check, I applied the same queries to both the original sample of 1000 IT projects and a larger sample of nearly 2200 IT projects. As the following chart shows, stratifying the data into size bins doesn't affect the overall direction of the trend:

For conventional productivity (FP/Person Month) the decline in productivity was even more pronounced:

Here are the results as of 3 pm today for our productivity poll. I was surprised to see that "Time slicing and multi-tasking" was far and away the most popular response!

When I framed the poll question, I was thinking of time slicing by policy (i.e., where management asks a single person to divide his time/attention between multiple projects). Viewed from a cost perspective, such assignments make perfect cents [pun fully intended] - especially for small or poorly capitalized projects. But from a productivity perspective, the constant distractions inherent in this kind of multi-tasking can result in less than optimal performance. 

Carol Dekkers made me think of another possible explanation for the declining productivity we've seen over the last 15 years: the distraction of 24/7 online communication. 

Since joining the world of social media I realize my “connectivity” has grown exponentially, but not all in a good way. Even with my SPAM filters set to high, I get so much email that it is overwhelming!

 I feel like I must have ADD (attention deficit disorder) because my day is interruption after interruption (sorry TweetDeck!) and I need help (and I know I am not the only one!)

If you're planning on attending this year's SEPG conference, don't miss QSM Consultant, researcher, and all around blogger extraordinaire Paul Below! On Wednesday, March 23rd, Paul will demonstrate how organizations can Maximize Value by Understanding the Productivity Paradox and Predicting Reliability:

Now more than ever, software projects need to efficiently deliver reliable software. However, many development plans unintentionally guarantee a less-than-optimal result. This presentation uses historical result metrics to describe how to maximize value by establishing minimum acceptable reliability and how to take advantage of the apparent paradox between software size and productivity through appropriate selection of team size and schedule duration.

If you've enjoyed Paul's posts here, make sure you stop by and say hello!

On our Software Productivity Declines Over Time post, ISBSG's Peter Hill left an interesting comment:

... your findings that 'average productivity increases with project size' might raise a few eyebrows, but the ISBSG data produces the same finding. Of course we only have data on projects that were completed. I wonder about the incidence of abandonment of large projects.

The QSM database shows a positive correlation between system size and productivity. The following chart shows PI vs. System Size regression fits for 9 different application domains (Business, System Software, Process Control, Telecom, Scientific, Command & Control, Avionics, Microcode, and Real Time). Regardless of domain, average productivity increases with system size:

View larger image

In the SLIM-Estimate user's guide, we speculate about possible explanations for the relationship between productivity and system size:

Small systems tend to have lower average PIs than large systems. We can only speculate as to the reasons, but two important influences may be management influence (the way large and small systems are planned, managed, and funded) and economies of scale.

Since we're speculating about why (in the aggregate at least) software productivity appears to have declined over time, here's another possible cause. I call it The Housework Theory of Productivity.

Back in great-Grandma's time, cleaning was a labor intensive endeavor. Rugs were swept once or twice a week and taken outside and beaten by hand once a year. Those cheery little scrubbing bubbles weren't around to whisk the pesky soap scum from our bathtubs - they hadn't been invented yet. When it came to cleaning, our aspirations were limited by the time and effort required to complete even the simplest tasks.

Over time various "labor saving" devices ushered in heretofore unheard of levels of springtime fresh cleanliness. But there was a downside to these advances: as our ability to get things clean increased, so did our expectations. The work expanded to fill the time (and energy) available to us.

If we were to compare two software projects that are same size and perform the same function (but were completed 20 years apart), we'd likely find the modern billing system smaller in size but far richer in features. It would also be more complex, both from an algorithmic and an architectural standpoint. Because today's programming tools and languages make it possible to deliver more value per line of code or function point, our standards have risen. We expect more for our time and money.

Peter Hill of ISBGS poses an interesting question:

Has software productivity improved over the last 15 years? What do you think? Perhaps it doesn't matter as long as quality (as in defect rate) as improved?

Two widely used productivity measures are Function Points/Person Month and QSM's PI (or productivity index). To answer Peter's question, I took a quick look at 1000 medium and high confidence business systems completed between 1996 and 2011. Here's what I found:

Whether we look at PI or FP/PM, the story's the same - on average, productivity has actually decreased over time.  What could be causing this? One possible explanation is the correlation between measured productivity and the volume of delivered functionality. As the next chart shows, regardless of the metric used average productivity increases with project size:

Which led me to wonder: what has happened to average project size over time? Again, regardless of whether the delivered functionality was measured in SLOC or Function Points, the story was the same: projects are getting smaller.

Peter's question is a good example of why we often need more than one metric to interpret the data. More on that topic coming up shortly!

It’s hard to believe it’s been 36 years since an IBM manager named Fred Brooks came out with his seminal insights about software development, the most famous of which ("adding more people to a late software project makes it later") came to be known as Brooks’ Law. These days, most software professionals accept and appreciate Brooks’ analysis, yet we continue to make the very mistakes that prompted him to write The Mythical Man Month!

Which leads to an interesting question: armed with such a clear and compelling argument against piling on staff at the last minute, why do we repeatedly employ a strategy that not only fails to achieve the hoped-for schedule reductions but often results in buggy, unreliable software?

The most likely answer combines schedule pressure with the human tendency to over-optimism. Basing plans on hope rather than experience is encouraged by a constant parade of new tools and methods. Faced with the pressure to win business, please customers and maintain market share, is it really surprising that new  technologies tempt us to discount the past and hope that – if we use this tool, this team, this methodology - this project will be different?

How can software developers counter the human tendency to fall for overly optimistic estimates and unachievable schedules?

What's needed is perspective: the kind of perspective that comes from honestly examining - and reminding ourselves - how things have worked in the past. In a paper called, “Technology Can Only Do So Much”, I look at the human and technological factors that trip up so many software projects.  Good historical data provides a sound empirical baseline, against which both conventional wisdom and future plans can be assessed.

What do you do if you want to improve a process and have 100 candidate predictor variables?  How do you decide where to direct causal analysis effort?  Similarly, what if you want to create an estimating model and you have so many factors you do not know where to start? Data mining techniques can be used to filter many variables down to a vital few to attack first or to build estimating models to predict important outcomes. CrossTalk Magazine, January 2011.

Read the full article here.

Paul Below has over 25 years’ experience in measurement technology, statistical analysis, estimating, forecasting, Lean Six Sigma, and data mining. He serves as a consultant for QSM, providing clients with statistical analysis of operational performance leading to process improvement and predictability.

Mr. Below is a Certified Software Quality Analyst, a past Certified Function Point Specialist, and a Six Sigma Black Belt. He has been a course developer and instructor for Estimating, Lean Six Sigma, Metrics Analysis, Function Point Analysis, as well as statistical analysis in the Masters of Software Engineering program at Seattle University. He has presented papers at numerous conferences. He has one US patent and two patents pending.