originally appeared in Vol. 10, no. 11 of AMERICAN PROGRAMMER.
©1998 by Cutter Information Corp. All rights reserved.
"SOFTWARE BY THE NUMBERS: AN
AERIAL VIEW OF
THE SOFTWARE METRICS LANDSCAPE"
by Michael C. Mah and Lawrence H. Putnam, Sr.
Foreword by Ed
With all of the new technologies and buzzwords in the computer field,
one might wonder why American Programmer returns to fundamental topics --
peopleware, project management, and the like -- so often. One reason is
that our industry still isn't doing a very good job at the fundamentals,
and another reason is that there always seems to be something new to say
about them. Such is the case with software metrics: it's been nearly two
years since we last covered the topic, and there are indeed some
interesting new things to say.
But Michael Mah and Larry Putnam (both from Quantitative Software
Management) remind us that we tend to get distracted by the gadgets and
buzzwords in the metrics field, just as we do in other aspects of
computing. As they point out, "A key point to consider on metrics is this:
rather than just musing on what 'new metric' might apply in the
ever-changing world of software, we should also be asking ourselves the
more basic question, 'What will we do with metrics?' . . . . The problem
oftentimes is that organizations don't have the basics down before they
pile on measure after measure." Mah and Putnam provide an excellent
summary of the basics; for organizations still in a state of "metrics
paralysis," their article is an excellent place to start.
You are erratic, conflicted, disorganized . . . You lack
harmony, cohesion, greatness. It will be your undoing.
-- Alien "Borg" character, addressing humans,
television series Star Trek: Voyager
Another playful take on this scenario might be that the "Borg" really
was delivering a process assessment using the Software Engineering
Institute's (SEI) Capability Maturity Model (CMM) and found the
organization at Level 1, otherwise known as "chaos."
All joking aside, no debate on the chaos and overruns of the software
industry is complete without discussion of "the M word," metrics. Whatever
the outcome, a key point to consider on metrics is this: rather than just
musing on what "new metric" might apply in the ever-changing world of
software, we should also be asking ourselves the more basic question,
"What will we do with metrics?" Then we can ask if the goals at hand
require any newfangled measures. The problem oftentimes is that
organizations don't have the basics down before they pile on measure after
Based on the answer to the "What will we do?" question, a clearer
purpose might arise, giving insight into where energies ought to be
channeled. One thing not to do is to measure everything in sight, hoping
to figure out what to do with the results later.
TURNING ON YOUR LIGHTS: GUIDANCE SYSTEMS THAT WORK
We manage things "by the numbers" in many aspects of our lives -- stock
market indexes, JD Power Quality figures for automobiles, Apgar scores for
newborns. These numbers give us insight and help steer our actions.
Software metrics extend the concept of "managing by the numbers" into the
realm of application development. That is their purpose.
Successful organizations have found three objectives extremely
valuable. These are:
- Knowing the capability, or productivity, of your organization. An
overused term, "benchmarking," has been used to describe this.
- Making credible commitments in terms of what will be delivered, when
it will be delivered, how much work effort will be needed, and how good
it will be when the system is placed into service. This involves project
estimation. Sometimes the "estimation" is how much functionality a team
can reasonably commit to build within a deadline that's been mandated by
external events (yes, estimation for "the real world").
- Managing development once it starts. This means making sure that the
commitments are on track to being accomplished. This involves project
management, but more than simple PERT and Gantt charts. It also requires
effective scope or size management and defect management.
Without accomplishing at least these objectives (and there are more
than just these three), a "metrics program" might lose perspective.
Reprinted by permission of United Feature
FASTEN YOUR SEATBELT, WE'RE ALREADY BEHIND SCHEDULE
Let's begin with the real world, complete with all its incredible
deadline pressure. The kind of world where an IBM executive was recently
quoted as saying -- facetiously, we hope -- "We either have to make our
deadlines, or kill our kids." Ugh!
We begin here because many organizations struggling with process chaos
claim that they don't have time for software measures, which are sometimes
perceived as a bureaucratic Dilbert exercise. But getting metrics in the
face of deadlines, and using them to manage that pressure and reduce the
chaos, is key to successful software development. Chaos with no measures
to tell the organization what's going on leads to studies that still show
3 in 10 projects being canceled, 5 in 10 overrunning schedule and/or
budget by nearly double, and only 1.6 in 10 making their deadlines and
budgets . Like it or not, Yourdon's noted "death march" projects are
sometimes more the rule than the exception, and sadly, the alien Borg
might have a point. Our chaos can be our undoing.
We're also seeing that it doesn't have to be that way.
Hopefully, interventions of some sort might break this cycle. In order
for that to happen, facts have to be known, communicated, and leveraged.
That is where the "mirrors" known as software metrics fit in. Anything
that doesn't help manage the pressure but might add to it (if seen as a
time waster) would be counterproductive.
WHAT METRICS TO FLY BY? START WITH "THE MINIMUM DATA SET"
Any discussion of metrics has to start with a foundation. Over the
years, a consensus has arisen to describe at least a core set of four
metrics. These are: size, time, effort, and defects. The SEI has issued a
useful publication that discusses the background to the core measures and
offers recommendations for their use . There are several additional SEI
documents available that go into further depth on the measures
individually. And lastly, prior to the SEI, some of the first writings on
the core set can be found in .
This "minimum data set" links management's bottom line in a cohesive
relationship. As project teams, we spend a certain amount of time
(months), expending a certain amount of work effort (person-months). At
the end of our hard work, the system is ready to be deployed (we hope). It
represents a certain amount of functionality (size) at a certain level of
quality (defects). Anyone embarking on a measurement program should start
with at least these four core measures as a foundation.
Why these four? Well, oftentimes, projects are managed by just two
metrics -- project milestones and effort (proportional to cost). This has
been described as akin to "flying the plane using only a watch and a fuel
gauge." Size and defects must be in the equation. They represent what has
been (or will be) built and the quality of the end result. In the end,
that is what has been created.
A good manager should be keeping these types of records. For
that have been completed, size represents what has been built, as
countable entities. Knowing what's been accomplished, at what speed, at
what cost, and at what level of quality can tell us how well we did. Think
"benchmarking," or, "knowing your capability."
In addition, it would be beneficial to add perhaps one or two more
metrics for past projects. One is the amount of rework. Another is the
degree of software reuse .
For projects that have yet to be built, the sizing issue becomes one of
estimation. The best way of approximating what needs to be built is to
have records about units you've built before, in order to help you scope
the new job. Size estimation is a critical discipline. It represents a
team's commitment as to what it will build. As Ed Yourdon once said:
Studies by the SEI indicate that the most common failing
of Level 1 (ad hoc) software organizations is an inability to make size
estimates accurately. If you underestimate the size of your next
project, common sense says that it doesn't matter which methodology you
use, what tools you buy, or even what programmers you assign to the
"NEW" JOURNEYS: SIZE METRICS FOR OO, CLIENT-SERVER, INTERNET APPS, AND
Whenever software developers begin a project that involves a new
technology (OO, client-server, etc.), there is great confusion as to how
and what they should measure and what the appropriate "size" unit might
be. The software development community has pondered questions like these
since what seems to be the beginning of time. You name the technology, the
language, the era. These questions often come down to size. Time we
understand. Effort we understand. Defects we understand (yet barely anyone
tracks them or keeps good records!). That last measure -- size -- is often
where all these questions lead.
For object-oriented development, useful measures of size have been
shown to be units such as number of methods, objects, or classes. Common
ranges seem to be about 175 to 250 lines of code (C++, Smalltalk, etc.)
per object. Lines of code, function points, classes, objects, methods,
processes, programs, Java scripts, and frames all represent various
abstractions of system size.
Leveraging these size units means taking an inventory of past
projects in order to better understand the building blocks of past systems.
This also establishes the vocabulary of the organization in terms of the
functionality that has been, and needs to be, built. It is the basis of
negotiation when trying to decide what a team agrees to take on, within a
Have Function Points Lived Up to Their Promises?
Of all the proponents of different size metrics, OO or otherwise, the
"priests" of function points have been the most insistent in promoting
that measure as the most important offering for all the altars of the
world. So, have function points lived up to their promises? It depends on
whom you ask.
In many organizations, function points seem to have served a useful
purpose. Organizations are finally getting people to think about project
size. It used to be that you'd ask the question, "How big is this
application?" and someone might answer, "150 man-months," answering your
question on size with a number for the effort (related to cost) spent
That's like someone asking you, "How big is your house?" and you
answer, "$250,000." You should have said something like, "It's a
four-bedroom colonial with 2 1/2 baths, living room, dining room, family
room, and den (10 rooms), for a total size of 2,400 square feet." High
abstraction unit: rooms; low abstraction unit: square feet.
Size is a metric describing the bigness or smallness of the system. It
can be broken into chunks of various descriptions. Function points can do
this in certain applications. As previously mentioned, other units include
programs, objects, classes, frames, modules, processes, computer software
configuration items (CSCIs in the world of DoD), subsystems, and others.
All represent the building blocks of the product from different levels of
abstraction, or perspectives. They all ultimately translate to the amount
of code that becomes compiled and built to run on a computer or embedded
They translate down to a common unit just as the volume of fluid in a
vessel might be described in terms of liters, gallons, quarts, pints, and
ultimately, down to a common unit of either fluid ounces or milliliters.
The key point to understand, though, is that all these units relate to
each other in a proportional, scaling relationship (i.e., 32 fluid ounces
per quart, four quarts per gallon).
So when questions like, "What new metrics should we use for . . . ?"
arise, gravitate to what would make sense in the organization's
vocabulary, if it comes down to project size. Remember that the objective
is to communicate information about the system and maximize the flow of
information by making sure everyone speaks a well-understood and familiar
language. Have language fit the organization, not the other way
Challenges in the Function Point World
For many DP/MIS organizations, the underlying concept of function
points is a decent fit. That is, the metamodel of a system comprising two
parts, a database structure and functions that access those structures,
correctly describes what they build. In this world, the latter functions
comprise Create, Read, Update, Delete (CRUD).
However, as Simon Moser and Oscar Nierstrasz rightly observe, "You have
a problem if your system does anything other than CRUD" . That's not
necessarily saying anything bad about the function point, other than that
it simply is not a size metric for all systems. Engineering applications,
factory automation, process control, and real-time systems are not IBM
back office-like mainframe systems that smoothly fit into the five
Attempts to massage function points into things like feature points,
data points, task points, and other points have therefore not been widely
accepted. So we'd also expect that any "object-oriented points,"
"client-server points," or "Internet points" that might come along in the
future would only represent additional efforts to make the CRUD metamodel
into something that it simply is not. Not everything is Create, Read,
Update, Delete against an underlying database.
Also, organizations have found metrics programs to be very valuable in
tracking and controlling projects already under way. (Think of a visual
analogy of an air traffic control tower, tracking "projects in the
air.") Unfortunately, function points can represent difficult entities to
track midstream. In ongoing projects, function point users have reported
difficulty in counting "what function points have been built so far." (On
the other hand, it's relatively easy for a configuration management system
to report how much code has been added as new and what code has been
changed.) In order to fill that void, many organizations respond by using
alternate size measures for tracking, such as modules built, number of
objects or programs under configuration management, number of integration
builds, and yes, amount of code built and tested to date.
So whether function points have met their early promise of consistency
in counting is up for debate. They may not have proved immune to counting
controversy after all. Problems of "fit" with the CRUD metamodel, dealing
with changed function points, midstream tracking, and the fact that no two
counts seem to come up the same are not uncommon.
Nevertheless, function points serve a purpose as one type of size
metric. And if your organization builds applications in the CRUD metamodel, it might pay to consider function points as a size metric.
Remember that any one measure has its limitations. Therefore, utilize
multiple approaches where possible and note the sizing relationships (the
proportionality) between them.
DEFECT METRICS: IS YOUR AIRCRAFT IN PROPER WORKING ORDER?
No metrics discussion would be complete without addressing the subject
of software defects. It is the least-measured entity, yet the one that
receives the worst press when software failures occur in the real
That should tell us something in and of itself. What we are seeing in
terms of "laws of nature" with software defects is the direct correlation
of schedule pressure and bad planning. Stan Rifkin, principal of Master
Systems Inc. in McLean, Virginia, summarized this once. He asked a
particular organization what had been a driving factor behind a 10x
reduction in defect levels. The response: effective planning.
This squarely points to the potential for management to influence
software quality, both positively and negatively. A chapter title of a new
software management handbook speaks volumes: "Managers Control Schedule,
and Influence Results Thereby" . In the chapter, the authors address
how defects rise due to schedule compression. This aspect of defect
behavior is independent and acts in addition to effects due to tools,
methodologies, and staff experience. Causality between software project
deadlines and defect levels thus places the opportunity for good quality
software squarely in our management laps.
Much has been said on the subject of which defect metrics to use. Two
categories deserve mention: (1) defects found during system testing to
delivery (including severity), over time, and (2) defects reported in the
first month (second, third, etc.) of service. The two categories are
inextricably related. The latter will enable you to determine software
reliability (say, in mean time to defect).
There is much to interpret from this raw data. At the heart of this is
causal analysis; that is, finding out the drivers behind defect rates and
defect densities and then doing something about them [4, 10].
TURNING ON ALL YOUR INSTRUMENTS
Now that measures exist, with the organization ready to move forward
with the best of intentions, the question remains of how to best use the
information at hand. In his newest book on information design, Visual
Explanations , Dr. Edward Tufte, professor of statistics and
information design at Yale University, opens by saying that assessment of
change, dynamics, and cause and effect are at the heart of thinking and
explanation. He further observes that proper arrangement in space of
images, words, and numbers requires a strategy for presenting
Developers, managers, and users need critical information. When they
don't get it, randomness takes over, leaving projects to drift into the
dynamics set in motion by tight deadlines, poor estimates, and volatile
requirements. This can be due to (1) information not being available in
the first place or (2) existing information being misrepresented or buried
in the details.
Two-Dimensional, Flatlands Views for a Multi-Dimensional Terrain
A most common information failure occurs if any of the four core
measures are reduced into overly simplistic two-dimensional ratios, such
as size over effort, or size over time. Some examples of these are
function points per person-month, lines of code per day, and so on.
Interpretation of these results can be very misleading. Why?
Software project data has revealed that, taken alone, neither ratio has
been reliable at giving the proper picture. For example, a project might
expend a great deal of work effort to achieve a remarkable schedule. Yet,
because schedule is not in the ratio of function points per work-month of
effort, that value will be low and will suggest only a nominal
Moreover, ratios suggest linear relationships, whereas software
development has been proven to be nonlinear. This means that you cannot
build a project in half the time with twice the people, although ratio
math would allow this, in theory. If you ever tried to bake a cake in 30
minutes at 650 degrees, you know that you'd get a burned brick. So if
baking a cake doesn't behave linearly, what would suggest that the complex
world of software development does, with all its team dynamics?
Using ratio math, we also risk portraying software development as a
production activity, like banging out widgets per day on an assembly line
or digging a ditch. But creating software is a research and development
activity. It involves a team coming up with ideas, and a technical
solution, for a problem domain. It involves trial and error, designing
some, coding some, testing some, and then possibly reworking it to get it
right. Two steps forward, one step back.
Indeed, when we examine curves for code progress, they never behave in
a straight line! They follow S-shaped rates of progression, starting off
slow during design, maxing out in the middle, and tapering way off during
testing, when little coding is being done. What's happening at that time
is a mad-scramble effort to get the bugs out, and rework the code, with a
So if you decide to examine ratios of functionality per unit effort, be
sure to look simultaneously at functionality per unit time and vice versa.
Be aware that they behave inversely and can vary sometimes by factors of 5
or even 10. For example, when schedules are compressed by adding staff,
effort goes way up. The ratio of functionality over effort goes way down.
But the functionality per unit time goes up, since time is shortened. This
happens all by itself due to staff size, in response to varying degrees of
schedule pressure. The behavior is independent of productivity and
process/environmental issues. In addition, expect to see high defect rates
when schedules are compressed in this manner. Therefore, don't forget to
An additional idea is to convert the SEI core measures into useful
indexes, or scales that are calculated. One such index is something known
as a software process Productivity Index (PI). The PI presents less risk
of dangerous linear interpretation. Best of all, its calculation is in the
public domain [6, 10]. The PI concept is straightforward. Higher values
represent projects built with less time, less effort, and better quality.
It is derived from the SEI minimum data set. Experiment with this and
other indexes to see how they might work for you.
Getting a Clear Navigational Picture
Misinformation may also arise due to political pressures, which result
in information being suppressed, deliberately altered, or misrepresented.
Tom DeMarco has called the latter "Limbaughing the data," which he defines
as, "To choose selectively from a body of data those items that confirm a
desired result and never mention any that might be construed to confirm
the opposite" .
This is a challenge about the politics of information (not something to
"Rush" into). It is an entirely separate subject needing much greater
representation than what is possible here. Let's exclude this to some
extent for now.
Let's assume that good information is available. Having metrics truly
serve their purpose requires solving the challenge of information design.
To that end, a picture is worth a thousand words. Busy managers already at
saturation need a visual, high-bandwidth message that transfers large
amounts of project and process information quickly and to the point with a
series of pictures. Where possible, try to craft metrics into visual media
that communicate your message effectively.
Some top-level questions include:
- Are the most critical projects on track? If not, then where are they
- What actions do we need to take? (Remember that earlier we discussed
asking the question, "What will we do with metrics?")
- Do our project estimates suggest that we have a high chance of
succeeding, or a high likelihood of failing, if we go that route?
Some other process-related questions include:
- What is our current capability?
- How do we compare in terms of speed, efficiency, and quality?
- Is our productivity improving?
In both areas, there is a great deal at stake in the answers, and thus
there is often the risk of "Limbaughing the data." However, assuming we
have all great intentions, communicating information effectively becomes
the metrics management challenge. Tables of numbers, simple
two-dimensional ratios of productivity, and chart clutter lose the
So the issue becomes one of making evidence and facts "visual," by
presenting information with images, color, motion, and so on. As Tufte
observes, the essence of quantitative thinking addresses the question,
"Compared to what?" Invalid comparisons must therefore be avoided like the
plague. And to understand the impact of actions, managers need metrics to
understand "cause and effect." Such as, "If we accept this deadline, and
we take on building this amount of functionality, how many bugs might
still be in the system on the date we'd like to ship?" (The demand for no
bugs at all is often an unrealistic demand in tight deadline
Risk management answers questions such as, "Do we have the odds against
us, or for us, when it comes to the deadline? When it comes to having
enough people? When it comes to the quality at delivery?"
When the answers to these questions are all negative, you could be
betting your job, and your company, as well as the customer. You are
pursuing a path of failure, not success. Rethink the strategy.
CASE STUDIES FROM FELLOW FREQUENT FLYERS
A major regional Bell telecom company with over 1,500 developers
identified an ad hoc software process as the underlying cause of software
rework, which drives up costs and lengthens schedules. Its goal of process
improvement therefore aimed to reduce rework. Effective use of software
metrics was key to that end, particularly better project estimation and
Over 100 project managers have been trained in advanced estimation
techniques, using models calibrated with the company's own past project
data. Senior managers attend regular "learning events" that deal with
maximizing use of information for risk management. The CIO has mandated
that the company's top 40+ projects all keep track of the four core
metrics, particularly size and defects. Information on the top projects is
assembled monthly on a "Management War Board." This high-order visual is
the basis of a "Software Control Tower," with its automated "virtual radar
screens" to identify position, direction, and expected arrival for all
incoming software project "flights."
For outsourced projects, vendor claims are being mapped to implied
productivity metrics, providing a starting point for contract
negotiations. The company is specifying defect metrics and reliability
goals, in addition to targets for delivered functionality at reduced
Moving from "Red Light" to "Green Light"
A real estate management software developer stamped out several
"software runaways" by getting management indicators for all projects
under way. The organization has put size metrics into place for
client-server and OO projects built in languages such as Visual C++ and
Visual Basic. Independent assessment of all critical projects
characterizes whether they are in a "red light," "yellow light," or "green
light" condition. Actions are taken on all projects that are yellow or
red. These assessments are conducted on a monthly basis for the executive
committee. Failing to submit data for analysis is not an option for any
On projects that are outsourced for competitive bid, all prospective
contractors answering requests for proposal (RFPs) are asked to provide
size, time, effort, and defect metrics on three or more recently completed
projects. Performance indexes are calculated and superimposed on industry
trends. The organization asks that project proposals include size
estimates in addition to proposed schedules and project costs. Bids are
assessed against past projects and industry benchmarks to assess their
validity. Contractors with the most credible proposals are chosen. They
may or may not be the ones with the lowest bids.
Recording and Learning from History
A Fortune 100 avionics system division with over 10,000 engineers uses
its own company intranet to serve as a repository and communication
vehicle for project metrics data. This division is one of two leading
worldwide developers of commercial and general avionics systems that fly
the planes built by Boeing, Airbus, and so on, as well as military
aircraft built by Lockheed Martin, General Dynamics, Northrop Grumman, and
Project data (including core metrics) and qualitative factors are
logged at a post-implementation review. This review occurs when the system
is delivered to the customer. At that time, all information is logged
while it is still fresh in the minds of the team members. A metrics group
services the division to provide data collection assistance, as well as
project estimation and control support.
Over 50 historic projects populate the division's database, and it's
still growing. Historic data is used to calibrate estimation models to
forecast scenarios for new projects and enhancement releases. Internal
experts have been certified to provide in-house company training courses on
project estimation and control.
Management has articulated goals for the metrics program, which include
reducing cycle time, maintaining quality, providing evidence and
credibility in bids for new business, and achieving higher CMM maturity
SOME WORDS FROM THE CAPTAIN AS WE APPROACH OUR DESTINATION
In a recent article discussing component-based development and
object-oriented design, software reuse expert Paul Bassett said that
organizations are at risk of being caught in "the forest [that] has grown
unduly bushy trees." The same risk applies when identifying and using
Keep it simple. Start with the four core metrics of size, time, effort,
and defects. Whatever you use for units, do it consistently. Don't
reinvent the wheel -- start with established standards such as those
articulated by the SEI. Once you have this information in place (for
several projects, one hopes), you'll get a bead on what Joe Kolinger of
Pacific Bell calls "knowing your capability."
Take it one day at a time. A metrics program is not a "boil the ocean"
project, to borrow a phrase from Tim Lister. Start with a small team of
dedicated individuals. Maybe it comes down to one or two people being
metrics champions at first. Get data on a few projects to start. Look
honestly and without fear when taking an inventory of software projects,
warts and all.
Ask yourself, What do we want to do with metrics? The answer will
likely translate down to managing commitments. What information will you
collect? And how will you communicate it?
Obtain Senior Management Support
Metrics confront management issues, such as negotiation of promised
functionality and the technical, cost, and quality viability of schedule
deadlines. Successful metrics programs are the ones in which a visionary
at the top supports the work of project managers and division heads.
If senior management's vision is fuzzy, uninformed, or misinformed,
then the captain of the plane has poor instruments and a mucky windshield.
Clarify this situation by dealing with management's bottom-line issues.
Some of these include managing schedule risk, quantifying the
organization's capability, and using metrics to win new business.
A Picture Is Worth a Thousand Words
Converting your numbers into pictures is key to getting out the
message. Innovative use of images, color, and motion makes for better
communication with less noise. So far, no one has been able to emulate Mr.
Spock's "Vulcan Mind Meld," so until then, we have to figure out better
ways of transferring high-bandwidth software information among teams,
managers, and end users. Edward Tufte's theories on information design
 are the wave of the future. Read his stuff.
And be careful about simple ratios of product over effort or product
over time. There is always the danger that such ratios will show only
partial dimensions; they are truly the two-dimensional flatlands in a
Reprinted by permission of United Feature
Beware the Hawthorne Effect and Heisenberg Uncertainty Principle
It's been said that to observe an event is to influence its outcome.
Criticism of metrics can be justified when measures are used improperly.
It might indeed be possible for bad metrics and bad data to drive out good
metrics and good data. Uninformed people might misinterpret numbers.
Hidden agendas can operate within an organization. And so on.
But as Tom DeMarco says, there is a bright side, in that problems like
these are indeed treatable with methods that we, as members of the
software metrics community, are fully competent to apply. DeMarco gives
valuable advice in this regard . These include things such as measuring
benefit and measuring for discovery. Other useful ideas are described by
Lawrence Putnam and Ware Myers in .
By all means measure to make good commitments. Measure to manage risk.
Risk is not bad, it is good. Taking unmitigated risks, not managing them
properly, and stacking the deck against oneself is bad.
Measure because of the Trucker's Maxim: "Behind every bouncing ball is
a running child." This notion applies to software projects as well.
Anticipate risk and be prepared. Software metrics should be applied to
keep our projects on the side of success. In some arenas, failure is not
1. Bassett, Paul G. Framing Software Reuse. Upper Saddle River, NJ:
Prentice Hall, 1997.
2. Carleton, Anita, Robert Park, and Wolfhart
Goethert. "The SEI Core
Measures: Background Information and Recommendations for Use and
Implementation. The Journal of the Quality Assurance Institute (July
3. DeMarco, Tom. Why Does Software Cost So Much, and Other Puzzles of
the Information Age. New York: Dorset House, 1995.
4. Grady, Bob. Practical Software Metrics for Project Management and
Process Improvement. Englewood Cliffs, NJ: Prentice Hall, 1992.
5. Johnson, Jim. "Chaos: The Dollar Drain of IT Project Failures."
Application Development Trends (January 1995), pp. 41-47.
6. Mah, Michael, and Lawrence Putnam. "Is There a Real Measure for
Software Productivity?" Programmer's Update (June 1990).
7. Moser, Simon, and Oscar Nierstrasz. "The Effect of Object-Oriented
Frameworks on Developer Productivity." IEEE Computer (September 1996), pp.
8. Putnam, Lawrence. Software Cost Estimating and Lifecycle Control:
Getting the Software Numbers. Los Alamitos, CA: IEEE Computer Society
9. Putnam, Lawrence, and Ware Myers. Executive Briefing: Controlling
Software Development. Los Alamitos, CA: IEEE Computer Society Press,
10. Putnam, Lawrence, and Ware Myers. Industrial Strength Software. Los
Alamitos, CA: IEEE Computer Society Press, 1997.
11. Tufte, Edward. Visual Explanations, Images and Quantities, Evidence
and Narrative. Cheshire, CT: Graphics Press, 1997.
Lawrence H. Putnam, Sr., is president of QSM,
Inc., a software metrics
firm that provides tools, education, and consulting for software
management. QSM is the developer of the SLIM Tool
Suite: SLIM-Estimate, SLIM-Control,
SLIM-DataManager, SLIM-MasterPlan and SLIM- Metrics,
leading-edge software measurement and estimation modeling tools used by
Fortune 500 and government agencies worldwide. QSM methods are used for
benchmarking, risk management, and "runaway" project prevention for both
outsourced and in-house software development.
Mr. Putnam can be reached at:
Quantitative Software Management
McLean, VA 22102
Phone: 703 790 0055; fax 703 749 3795
Web site: http://www.qsm.com.
Mr. Mah can be reached at QSM Associates, Inc.,
Clock Tower Business
Park, 75 South Church Street, Pittsfield, MA 01201 (+413 499 0988; fax
+413 447 7322; e-mail: firstname.lastname@example.org; Web: http://www.qsma.com/.)
Reprinted from Vol. Vol. 10, no. 11 of AMERICAN PROGRAMMER. © Copyright
1998 by Cutter Information Corp., 37
Broadway, Arlington, MA 02174, USA. Phone: (781) 648-8702 or (800)
964-8702, Fax: (781) 648-1950, E-mail: email@example.com, Web site: www.cutter.com/itgroup/. All