a stitch in time saves nine Economics of defect prevention

2
a stitch in time saves nine Economics of defect prevention

Abstract
Do you think high quality means more money and more time? Many
organizations think so.
In a way it is true for the simple fact that one half of most software
development project budgets and two thirds of the typical development
teams time are spent fixing poor quality or software defects. A study
conducted by National Institute of Standards and Technology (NIST)
reports that software defects cost the U.S. economy $59.5 billion annually.
Caper Jones in his 2001 IEEE paper, Measuring Software Process
Improvement, gave a hope to software executives around the world by
revealing that software process improvement leads to 80% reduction in
post-release defects and a 65% increase in productivity, while reducing
project schedules and costs by 20%.
This paper highlights the need for an early detection of defects, its impact
on the business & also emphasizes on the cost of fixing a defect.

Introduction - Cost of Poor Quality is nothing

but Defects:
An important strategy of any IT organization is to deliver defect free or
quality products. But developing the quality product comes with a price
tag.
Dr. Joseph Juran, the father of the quality developed the Cost of
Quality concept (figure1) a measure that takes into account all the costs
associated with good and poor quality. In essence the cost of quality
represents the costs to achieve good quality as well as the costs incurred
due to poor quality.

Figure 1

Cost of poor quality attributes to internal failure costs and external

failure costs.
?
Internal failure costs - the cost incurred due to the defects, before

shipping the products to production or live environment. This is preproduction cost.

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

3
a stitch in time saves nine Economics of defect prevention

?
External failure costs - the cost incurred due to the defects, after

shipping the products to production or live environment. This is postproduction cost.

Its important to correctly classify quality costs to determine how much
poor quality is costing your organization.
Not all defects in the software development are caused by coding errors.
Some examples of defects coming upstream in the software development
lifecycle (SDLC) are defects due to misunderstanding of requirements,
contradicting requirements, missing requirements, wrong analysis of
requirements, incomplete requirements and more importantly unable to
identify non-functional requirements such as testability, scalability,
maintainability, usability, performance etc.
A programmer makes a mistake, which results in a defect in the software
source code. If this defect is executed, in certain situations the system will
produce false results, causing a failure. However all defects will not
necessarily result in failures. For example, defects in dead code will never
result in failures. A defect can turn into a failure when the environment is
changed. For example, software being run on a new hardware platform,
changes in source data or interacting with different software. A single
defect may result in a wide range of failure symptoms.

Why defects need to be detected and

fixed early? :
What industry trends on defects say:
Industry trends indicate that though 56% of the defects are injected in
requirements phase, most of these defects are actually detected in
integration / system testing phase only. The longer the defect stays in the
program, the more likely to have additional code written around it, the
more re-testing to be done and incurring additional cost to fix. So it is
evident that it is always less expensive to detect and fix the defects early in
SDLC than fixing in the production.
Industry trends on defects injection

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

4
a stitch in time saves nine Economics of defect prevention

Industry trends on defects detection

Figure 2: Source: Extracts from chapter 5 from the National Institute of

Standards and Technology (NIST) Report on The Economic Impacts of
Inadequate Infrastructure for Software Testing.

Relative cost of defects & its impact on the business:

The concept of cost of fixing a defect is not well understood.

Figure 3

This cost feeds back into SDLC and many managers do not actually know
what it costs their organizations to fix the defects.
An estimate by Boehm & Basili suggests that the cost of fixing the defect
grows exponentially for each downstream phase of the software
development lifecycle in which it remains undetected. The fixing of the
defect after product delivery can be 100 times more costly than doing so in
the requirements and design phases.

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

5
a stitch in time saves nine Economics of defect prevention

Table 1 also shows the cost of fixing the defect depending on the SDLC phase & the time it is detected. If
the defect in the requirements is detected post-release, then it would cost 10100 times more to fix than if the
defect had already been detected in the requirements phase.

Cost to Fix a Defect

Time Detected
Requirements Architecture Construction System testing Post-release

Time
Introduced

Requirements

510

10

10100

Architecture

10

15

25100

Construction

10

1025

Table 1

Table 2 indicates some real time impacts of software defects on the

business:
Business

Cost per Hour Downtime

Automated teller machines (medium sized bank)

$14,500

Package shipping service

$28,250

Telephone ticket sales

$69,000

Catalog sales center

$90,000

Airline reservation center (small airline)

$89,500

Table 2: Source :Information Technology Project Management, Fourth Edition

Calculation of Cost of Fixing a Defect:

Formula:
Average cost
(Number of resources X Duration to fix the defects)
to fix a defect = --------------------------------------------------------- X Cost per person-day

(Number of defects fixed)

Below you would find two hypothetical cases of a project that will help
you understand how to determine the cost of fixing a defect & its impact
in your project.
Project X is developing a financial software product and below are the
project details:
Duration

11 months

Team Size

10 FTE

Scope

Requirements, Design, Construction, Unit Testing and System Testing

Billing Rate

$500 per person-day

Table 3

Project has spent initial 10 months (as per estimated schedule) in

developing the product. Now the project
is in the system testing phase.

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

6
a stitch in time saves nine Economics of defect prevention

Before: All defects are detected in system testing

The project has detected 300 defects in system testing phase and zero
defects in previous phases. Refer figure 4.

Figure 4

Before Scenario

Cost of fixing the defects in system testing:

# of resources deployed

10

# of valid defects detected

300

# of defects fixed

300

Duration to fix the defects

60 days

Effort to fix the defects

10*60=

Bef
ore
Sce
nar
io

600 Person days

Defect fixing rate

2 person days per fix

Billing rate

$500 per person-day

Average cost to fix a defect in

system testing

500*2=$1000 per fix

Table 4

System testing defects cost: 1000*300 = $300,000. This is preproduction cost.

During post production stage, Project X has spent on an average 5
person-days per fix. At $500 per person-day, the cost per fix is $2500.
Project X has released 50 fixes, Total post-release fix cost = $125,000. This
is post- production cost.
Total cost incurred for fixes or defects (pre production +post
production) = $425,000 + Indirect cost*.
(Indirect Cost* - system unavailability, service loss, environment setups,
technology complexity, project management effort, QA effort etc).

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

7
a stitch in time saves nine Economics of defect prevention

After: Defects detected & fixed throughout SDLC

The project has detected and fixed 180 defects in the previous phases and
remaining 120 defects are detected in system testing. Refer figure 5.

Figure 5

After Scenario

Cost of fixing the defects in system testing:

# of resources deployed

10

# of valid defects detected

120

# of defects fixed

120

Duration to fix the defects

20 days

Effort to fix the defects

10*20=
200 Person days

Defect fixing rate

1.6 person days per fix

Billing rate

$500 per person-day

Average cost to fix a defect in

system testing

500*1.6=$800 per fix

Table 5

System test defects cost: 800*120 = $96,000 +RDC Cost*.

(RDC Cost* - Requirements, Design, Construction & Unit Testing (RDC)
defects cost should also be calculated same as above table5).
For the calculation purpose, lets assume the RDC cost incurred is 30%
system testing defects i.e $28,800.
System Test defects cost: $124,800. This is pre-production cost. Post
production cost is NIL.
Total cost incurred for fixes or defects (pre production +post
production) = $124,800.
In the after scenario, project X has saved over $300K in addition
to saving associated with indirect costs. Other benefits include
cycle time reduction, achieving project schedule, better
productivity and customer confidence.

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

8
a stitch in time saves nine Economics of defect prevention

Conclusion
It takes courage and commitment to make a Zero defect policy. But with
focused effort one can certainly commit to achieving minimal defects and
reduce the cost associated with them. Product managers can absolve
themselves from the dilemma of whether we need to do one more round
of testing. Defect prevention simplifies the test cycles, reduces overall
cost and improves product quality.

About the Author:

SUDHEER CHIKKERUR is a senior process consultant at HCL
Technologies Limited.
HCLs IV&VS - Quality Consulting Group works with client
organizations on software process improvements to improve product
quality while reducing operational cost. For more information contact us
at process.consulting@hcl.com

References:
Boehm B, Basili VR. Software defect reduction top 10 list. IEEE
Computer. January, 2001.
Principles of Quality Costs: Principles, Implementation, and Use,Third
Edition, ed. Jack Campanella, ASQ
What Does It Cost to Fix a Defect? By Johanna Rothman

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.

9
a stitch in time saves nine Economics of defect prevention

ABOUT HCL
HCL Technologies
HCL Technologies is a leading global IT services company, working with
clients in the areas that impact and redefine the core of their businesses.
Since its inception into the global landscape after its IPO in 1999, HCL
focuses on transformational outsourcing, underlined by innovation and
value creation, and offers an integrated portfolio of services including
software led IT solutions, remote infrastructure management,
engineering and R&D services and BPO. HCL leverages its extensive
global offshore infrastructure and network of offices in 31 countries to
provide holistic, multi-service delivery in key industry verticals including
Financial Services, Manufacturing, Consumer Services, Public Services
and Healthcare. HCL takes pride in its philosophy of Employees First
which empowers our 72,267 transformers to create real value for
customers. HCL Technologies, along with its subsidiaries, had
consolidated revenues of US$ 3.1 billion (Rs. 14,101 crores), as on 31st
December 2010 (on LTM basis). For more information, please visit
www.hcltech.com

About HCL Enterprise

HCL is a $5.7 billion leading global technology and IT enterprise
comprising two companies listed in India - HCL Technologies and HCL
Infosystems. Founded in 1976, HCL is one of India's original IT garage
start-ups. A pioneer of modern computing, HCL is a global
transformational enterprise today. Its range of offerings includes product
engineering, custom & package applications, BPO, IT infrastructure
services, IT hardware, systems integration, and distribution of
information and communications technology (ICT) products across a
wide range of focused industry verticals. The HCL team consists of over
79,000 professionals of diverse nationalities, who operate from 31
countries including over 500 points of presence in India. HCL has
partnerships with several leading Global 1000 firms, including leading IT
and Technology firms. For more information, please visit www.hcl.com

2011, HCL Technologies. Reproduction Prohibited. This document is protected under Copyright by the Author, all rights reserved.