SPECIAL TOPICS IN

Information
Technology

SOFTWARE TESTING
AJ Rosal

WHAT IS A COMPUTER BUG

In 1947 Harvard University was

operating a room-sized computer
called the Mark II.
A moth flew into the computer and
was zapped by the high
voltage
when it landed
on a relay.
Hence, the first computer bug!

BUGS AKA

Defect
Problem
Error
Incident
Anomaly
Variance

Failure
Inconsistency
Product
Anomaly
Product
Incidence

SOURCES OF PROBLEMS
Requirements Definition: Erroneous,
incomplete, inconsistent requirements.
Design: Fundamental design flaws in the
software.
Implementation: Mistakes in chip
fabrication, wiring, programming faults,
malicious code.
Support Systems: Poor programming
languages, faulty compilers and debuggers,
misleading development tools.

A software bug occurs when at least one of these rules is true

o The software does not do something that the

specification says it should do.
o The software does something that the
specification says it should not do.
o The software does something that the
specification does not mention.
o The software does not do something that the
product specification does not mention but
should.
o The software is difficult to understand, hard to
use, slow

RELATIVE COST OF BUGS

Cost to fix a bug increases exponentially

(10x)
i.e., it increases tenfold as time increases

E.g., a bug found during specification costs

$1 to fix.
if found in design cost is $10
if found in code cost is $100
if found in released software cost is
$1000

BUG FREE SOFTWARE

Software is in the news for the wrong

reason
Security breach, Mars Lander lost, hackers
getting credit card information, etc.

Why cant software engineers develop

software that just works?
As software gets more features and supports
more platforms it becomes increasingly
difficult to make it create bug-free.

GOAL OF A SOFTWARE TESTER

to find bugs
as early in the software
development processes as possible
and make sure they get fixed.
Advice: Be careful not to get get
caught in the dangerous spiral of
unattainable perfection.

GOAL OF A SOFTWARE TESTER

to find bugs
as early in the software
development processes as possible
and make sure they get fixed.
Advice: Be careful not to get get
caught in the dangerous spiral of
unattainable perfection.

SOFTWARE TESTING

Testing is the
process of
establishing
confidence that
a program does
what it is
supposed to do.

SOFTWARE TESTING

Testing is the
process of
demonstrating
that errors are
not present.

SOFTWARE TESTING

SOFTWARE is 99%
ERROR FREE

SOFTWARE TESTING

The purpose of
testing is to
show that a
program
performs its
intended
functions
correctly

SOFTWARE TESTING

WHAT IS THE
BEST TESTING
PROCEDURE?

SOFTWARE TESTING

A good test case

is one that has a
high probability
of finding an
asyetundiscovered
error.

THE BUGS

SOFTWARE BUGS

Defect categories
Wrong
The specifications have been
implemented incorrectly.
Missing
A specified requirement is not
in the built product.
Extra
A requirement incorporated
into the product that was not
specified.
Srihari Techsoft

History of Software Bugs

TRIVIA
The term Computer Bug is coined by
Grace Murray Hopper.
During her work on Mark II and Mark III
machine was facing some problem, later
operators investigated and identified that
there was a moth (Insect) trapped in a
relay which was causing problem.

TRIVIA
This Computer Bug (Insect) removed and taped
to log book September 9th 1945 at 3:45 p.m.
First, actual case of bug being found in
computer.
Grace Murray Hopper is also a Mother of COBOL.
The word went out that they had debugged
the machine and the term debugging a
computer program was born

TRIVIA

SOFTWARE TESTING
SOFTWARE TESTING
Non Functional Testing
Testing Methods

1. White box testing

2. Black box testing
3. Grey box testing

Testing Level

1.
2.
3.
4.
5.
6.
7.
8.

Unit testing
Integration testing
System testing
System integration
testing
Regression testing
Acceptance
testing
Alpha testing
Beta testing

Non Functional Testing

1. Software
performance
testing and load
testing
2. Stability testing
3. Usability testing
4. Security testing
5. Internationalization
and localization
6. Destructive testing

CONVENTIONAL SOFTWARE TESTING

Software product meets specifications.
Software performs all the functions
called for in the software requirements
specifications (SRS)
Intent of finding errors
Determines that it meets its required
results

CONVENTIONAL SOFTWARE TESTING

Usability
Functionality
e.g., Test the accurate
workings of each
usage scenario

Supportability
e.g., Test the ability to
maintain and support
application under
production use

e.g., Test application from

the perspective of
convenience to end-user.

Reliability
e.g., Test the application
behaves consistently and
predictably.

Performance
e.g., Test online
response under average
and peak loading

THE NEW PARADIGM - DST

The proposed testing paradigm
tests to see if a software product
exhibits proper behavior when
subject to improper usage or
improper input.

DESTRUCTIVE SOFTWARE TESTING

DST does not conform to any of the previous
definitions proposed.
DST can be performed without knowledge of
the original requirements of a software product.
Although some knowledge of the
requirements may help in developing a good
comprehensive destructive testing strategy.

HOW IS IT DONE
Injecting faults into production systems
(services, servers, and network) to validate
service continuity in the event of a real fault.
The random and unexpected occurrence of
such faults is a certainty in any service of
substantial scale.

PURPOSE OF DST
Prolonged endurance testing under the
most
severe
operating
conditions,
continued until the component, equipment,
or product fails (is broken or destroyed).
The purpose of destructive testing is to
determine service life and to detect design
weaknesses that may not show up under
normal working conditions.

BASIC PRINCIPLE OF DST

Is not a replacement for conventional testing.
Should be performed in addition to
conventional testing.
Acknowledges the fact that users of a
software product will sometimes not use the
software correctly.
Software Misuse

BASIC PRINCIPLE OF DST

Improper or incorrect
Input data will be supplied
Letters entered when numbers are
expected.

Commands will be typed

Commands that the software was
not written to handle.

GUI sequences will be applied

Alternate path that wasnt
expected.

EXAMPLE
In one year, Google expects to see 20
rack failures, three router failures and
1000s of server failures. So if these
failures are sure to occur, it is the
testers duty to assure the service can
handle them when they do

Introduction
Conventional Software Testing Definitions
the activity or process which shows or
demonstrates that a program or system
performs all intended functions correctly
the activity of establishing the necessary
confidence that a program or system does
what it is supposed to do, based on the
specifications and requirements set by the
user

Destructive Software Testing

(DST)
DST does not conform to any of the
previous definitions proposed.
DST can be performed without
knowledge of the original
requirements of a software product.
Although some knowledge of the requirements
may help in developing a good comprehensive
destructive testing strategy.

Basic Principles of DST

Is not a replacement for conventional
testing.
Should be performed in addition to
conventional testing.
Acknowledges the fact that users of a
software product will sometimes not
use the software correctly.
Software Misuse

Software Misuse
Improper or incorrect
Input data will be supplied
Letters entered when numbers are
expected.

Commands will be typed

Commands that the software was not
written to handle.

GUI sequences will be applied

Alternate path that wasnt expected.

Destructive Testing Adage

An popular adage
garbage in, garbage out
This adage is not good enough for high quality,
robust, and reliable software.

A better adage for DST

garbage in, proper predictable behavior out.

DST improves the quality of a software

product.

Destructive Testing
Terminology
Destructive Hardware Testing (DHT)
Hardware systems are destroyed as part
of testing
Example: Automobile safety testing
The usual practice is to subject the automobile to
an actual accident where the vehicle is heavily
damaged or destroyed

DST was chosen to be in compliance

with corresponding concept of DHT

DST Definitions
DST is testing that
assures proper or predictable software
behavior when the software is subject to
improper usage or improper input
attempts to crash a software product
tries to crack or break a software
product
checks the robustness of a software
product

Destructive Testing Confusion

The Term Destructive Testing
Should not be confused with the same term
used for conventional software testing
Has been used in the past to indicate
software that fails conventional testing

Conventional Testing
Strategies to DST
Strategies that cannot be used for
DST
Verification Testing
Validation Testing
Benchmark Testing

Incorporation of DST into

Specifications
Requirements for a software system can
be written to promote DST
Such requirements, by definition, are nonfunctional
Functional requirements by nature fall into the
category of conventional testing that can not
be tested using destructive testing
Verification Testing
Validation Testing
Benchmark Testing

Incorporation of Destructive
Testing
into Specifications
Mandatory destructive testing
clauses

The software must be designed in such

a way that proper behavior is obtained
when the software is subject to improper
usage or improper input.
No data shall be lost if the software
prematurely terminates as a result of
other system failures.
The software shall not prematurely or
unintentionally terminate as a result of
any combination of user keyboard or
mouse input.

Incorporation of Destructive
Testing
into Specifications

Mandatory destructive testing clauses

The software shall never accept or
process invalid input data.
The software shall always produce
proper output data regardless of the
validity or correctness of input data.

Steps for testing

1.
2.
3.
4.
5.

Plan user tests

Conduct user tests
Analyze findings
Present findings
Modify and retest designs

Incorporation of Destructive
Testing
into Specifications

For Specific software products, it is

important to explicitly define

proper software behavior

improper software behavior
improper usage
improper input data
proper output data

Future work
Author is currently working with a team
to develop requirements specification
for an example case study involving a
data conversion program.
Subsequent to the completion of the
requirements specification and
implementation of the software, test
cases will be developed for destructive
testing of the software.

Conclusion
The goal of conventional software testing is to
ensure a software product correctly performs
all the functions specified in the requirements
specification.
The goal of destructive testing is to ensure a
software product exhibits proper behavior
when subject to improper usage or improper
input.
Ongoing work includes the development of
requirements specification that mandate
destructive testing of a case study software
product.

Conclusion
Destructive testing is a reflection of the
fact that user will sometimes use a
software product in an improper manner.
Destructive testing does not replace
conventional testing
Supplements conventional testing
Performed in addition to conventional testing

Destructive testing cannot be detrimental

Destructive testing can only be beneficial.

The 7Deadly Sins in S/W Testing

Questions