Chapter 2

Foundations of Software Testing
Chapter 2: Test Generation: Requirements

Aditya P. Mathur
Purdue University
These slides are copyrighted. They are for use

with the Foundations of Software Testing
book by Aditya Mathur. Please use the slides
but do not remove the copyright notice.
Last update: October 6, 2010

Learning Objectives
  Equivalence class partitioning

Essential black-box techniques to
  Boundary value analysis generate tests for functional testing.
  Test generation from predicates
© Aditya P. Mathur 2006

2
Applications of test generation
techniques
Test generation techniques described in this chapter belong to the

black-box testing category.
These techniques are useful during functional testing where the objective
is to test whether or not an application, unit, system, or subsystem,
correctly implements the functionality described in the requirements

3
Functional Testing: Test Documents
(IEEE829 Standard)
Reference: Lee Copland.
Requirements Model A Practitioner’s Guide to Software Test
Design
Test Plan Test Design Test Case Test Procedure

Spec. Spec.
Test item transmittal Test log Test incident

report report
Test summary
Test generation techniques report
4
Functional Testing: Documents
Test Plan: Describe scope, approach, resources, test schedule,
items to be tested, deliverables, responsibilities, approvals needed.
Can be used at the system test level or at lower levels.
Test design spec: Identifies a subset of features to be tested and

identifies the test cases used to test the features in this subset.
Test case spec: Lists inputs, expected outputs, features to be

tested by this test case, and any other special requirements,
such as setting of environment variables and test procedures.
Dependencies with other test cases are specified here. Each
test case has a unique ID for reference in other documents.
5
Functional Testing: Documents
(contd)
Test procedure spec: Describes the procedure for executing a test
case.
Test transmittal report: Identifies the test items being provided for
testing, e.g. a database.
Test log: A log of the observations during the execution of a test.
Test incident report: Document any special event recommended for

further investigation.
Test summary: Summarizes the results of testing activities and

provides an evaluation.
6
Test generation techniques in this
chapter
Four techniques are considered:
•  equivalence partitioning
•  boundary-value analysis
•  cause-effect graphing (not in the foils)
•  predicate-based test generation
Each of these test generation techniques is a black-box technique

useful for generating test cases during functional testing.

7
The test selection problem
8
Requirements and test generation
Requirements serve as the starting point for the generation of tests.

During the initial phases of development, requirements may exist only
in the minds of one or more people.
These requirements, more aptly ideas, are then specified rigorously

using modeling elements such as use cases, sequence diagrams,
and statecharts in UML.
Rigorously specified requirements are often transformed into formal

requirements using requirements specification languages such as Z,
S, and RSML.

9
Test generation techniques

10
Test selection problem
Let D denote the input domain of a program P.
The test selection problem is to select a subset T of tests such that
execution of P against each element of T will reveal all errors in P.
In general, there is no algorithm to build such a test set.
However, there are heuristics and model-based methods that can be

used to generate tests that will reveal all faults of a certain type.
The challenge is to build a test set T ⊆ D that will reveal as many

errors in P as possible.
The problem of test selection is difficult due to the size and complexity
of the input domain of P.
11
Exhaustive testing
The large size of the input domain prevents a tester from exhaustively
testing the program under test against all possible inputs.
By “exhaustive” testing we mean testing the given program against

every element in its input domain.
The huge size of the input domain also makes it hard to select individual
tests.

12
Large input domain
Consider program P required to sort a sequence of integers into
ascending order.
Assuming that P will be executed on a machine where integers range

from -32768 to 32767, the input domain of P consists of all possible
sequences of integers in the range [-32768, 32767].
If there is no limit on the length of the sequence that can be input,

the input domain of P is infinite and P cannot be tested exhaustively.
If the size of the input sequence is limited to Nmax>1, then the size of
the input domain depends on the value of Nmax.
Calculate the size of the input domain.

13
Complex input domain
Consider a procedure P in a payroll processing system, which takes an
employee record as input and computes the weekly salary.
For simplicity, assume that the employee record consists of the

following items with their respective types and constraints:
Calculate the size of the input domain.

14
Equivalence class partitioning
15
Equivalence partitioning
Test selection using equivalence partitioning allows a tester to subdivide
the input domain into a relatively small number of sub-domains, say N>1.
The sub-domains by definition are disjoint. The four subsets shown in (a)
constitute a partition of the input domain while the subsets in (b) are not.
Each subset is known as an equivalence class.

16
Program behavior and equivalence
classes
The equivalence classes are created assuming that the program

under test exhibits the same behavior on all elements, i.e. tests,
within a class.
This assumption allow the tester to select exactly one test from
each equivalence class resulting in a test suite of exactly N tests.

17
Faults targeted
The entire set of inputs to any application can be divided into at least two
subsets: one containing all the expected, or legal, inputs (E) and the other
containing all unexpected, or illegal, inputs (U).
Each of the two subsets can be further subdivided into subsets on which the
application is required to behave differently (e.g. E1, E2, E3, and U1, U2).
Equivalence class partitioning selects tests that

target any faults in the application that cause it
to behave incorrectly when the input is in either
of the two classes or their subsets.

18
Example 1
Consider an application A that takes an integer denoted by age as input.
Let us suppose that the only legal values of age are in the range [1..120].
The set of input values is now divided into a set E containing all integers in
the range [1..120] and a set U containing the remaining integers.
All integers
Other integers
[1..120]

19
Example 1 (contd.)
Further, assume that the application is required to process all values in the
range [1..61] in accordance with requirement R1 and those in the range
[62..120] according to requirement R2.
E is further subdivided into two regions depending on the expected behavior.
Similarly, it is expected that all invalid inputs less than or equal to 1 are to
be treated in one way while all greater than 120 are to be treated differently.
U is further subdivided two categories.
All integers
<1
[62-120] >120
[1..61]
20
Example 1 (contd.)
Tests selected using the equivalence partitioning technique aim at targeting
faults in the application under test with respect to inputs in any of the four
regions, i.e. two regions containing expected inputs and two regions
containing the unexpected inputs.
It is expected that any single test selected from the range [1..61] will
reveal any fault with respect to R1.
Similarly, any test selected from the region [62..120] will reveal any fault
with respect to R2.
A similar expectation applies to the two regions containing the unexpected
inputs.

21
Effectiveness
The effectiveness of tests generated using equivalence partitioning for

testing application A is judged by the ratio of the number of faults these
tests are able to expose to the total faults lurking in A.
As is the case with any test selection technique in software testing, the
effectiveness of tests selected using equivalence partitioning is less than 1
for most practical applications.
The effectiveness can be improved through an unambiguous and complete

specification of the requirements and carefully selected tests using the
equivalence partitioning technique described in the following sections.

22
Example 2
This example shows a few ways to define equivalence classes based on

the knowledge of requirements and the program text.
Consider a wordCount method which takes a word w and a filename f as

input and returns the number of occurrences of w in the text contained
in the file named f.
An exception is raised if there is no file with name f.

23
Example 2 (contd.)
begin
String w, f
Input w, f
if (not exists(f) {raise exception; return(0);}
if(length(w)==0) return(0);
if(empty(f)) return(0); Using the partitioning method described
in the examples above, we obtain the
return(getCount(w,f)); following equivalence classes.
end
24
Example 2 (contd.)
Equivalence class w f
E1 non-null exists, not empty
E2 non-null does not exist
E3 non-null exists, empty
E4 null exists, not empty
E5 null does not exist
E6 null exists, empty

25
Example 2 (contd.)
The number of equivalence classes without any knowledge of

the program code is 2, while the number of equivalence classes
derived with the knowledge of partial code is 6.
Of course, an experienced tester will likely derive the six equivalence

classes given above, and perhaps more, even before the code is available

26
Equivalence classes based on
program output
In some cases the equivalence classes are based on the output generated
by the program. For example, suppose that a program outputs an integer.
It is worth asking: “Does the program ever generate a 0? What are the
maximum and minimum possible values of the output?”
These questions lead to two the following equivalence classes:

•  E1: Output value v is 0.
•  E2: Output value v is the maximum possible.
•  E3: Output value v is the minimum possible.
•  E4: All other output values.
Based on the output equivalence classes one may now derive equivalence
classes for the inputs. Thus each of the four classes given above might
lead to one equivalence class consisting of inputs.
27
Equivalence classes for variables:
range
Equivalence Classes Examples

Constraints Classes
One class with values inside speed ∈ [60..90] {{50}, {75}, {92}}
the range and two with
values outside the range.
area: float {{-1.0}, {15.52}}

area≥0.0
age: int {{-1}, {56}, {132}}
letter: bool {{J}, {3}}

28
strings
Equivalence Classes Example

Constraints Classes
At least one containing firstname: string {{ε}, {Sue},
all legal strings and {Looooooong Name}}
one all illegal strings
based on any
constraints.

29
enumeration

Constraints Classes
Each value in a color: {red, blue, green} {{red,} {blue}, {green}}
separate class
up: boolean {{true}, {false}}

30
arrays
Constraints Classes
One class containing int [ ] aName: new int {[ ]}, {[-10, 20]}, {[-9,
all legal arrays, one [3]; 0, 12, 15]}
containing the empty
array, and one
containing a larger
than expected array.

31
compound data type
Arrays in Java and records, or structures, in C++, are compound types.

Such input types may arise while testing components of an application
such as a function or an object.
When generating equivalence classes for such inputs, we must consider
legal and illegal values for each component of the structure.
struct transcript {
string fName; // First name.
string lName; // Last name.
string cTitle [200]; // Course titles.
char grades [200]; // Letter grades corresponding to course titles.
}

32
Unidimensional partitioning
One way to partition the input domain is to consider one input variable
at a time.
Thus, each input variable leads to a partition of the input domain.
We refer to this style of partitioning as unidimensional equivalence

partitioning or simply unidimensional partitioning.
This type of partitioning is commonly used.

33
Multidimensional partitioning
Another way is to consider the input domain I as the cross product of the
domains of the input variables and define a relation on I.
This procedure creates one partition consisting of several equivalence classes.

We refer to this method as multidimensional equivalence partitioning or
simply multidimensional partitioning.
Multidimensional partitioning leads to a large number of equivalence

classes that is difficult to manage manually.
Many classes so created might be infeasible.
Nevertheless, equivalence classes so created offer an increased variety of
tests as is illustrated in the next section.
34
Partitioning Example
Consider an application that requires two integer inputs x and y,
expected to lie in the following ranges: 3≤x≤7 and 5≤y≤9.
For unidimensional partitioning we apply the partitioning guidelines to x

and y individually. This leads to the following six equivalence classes.
E1: x<3 E2: 3≤x≤7 E3: x>7 y ignored.
E4: y<5 E5: 5≤y≤9 E6: y>9 x ignored.
For multidimensional partitioning we consider the input domain to
be the cross product of x and y. This leads to 9 equivalence classes.
E1: x<3, y<5 E2: x<3, 5≤y≤9 E3: x<3, y>9
E4: 3≤x≤7, y<5 E5: 3≤x≤7, 5≤y≤9 E6: 3≤x≤7, y>9
E7: x>7, y<5 E8: x>7, 5≤y≤9 E9: x>7, y>9
35
Partitioning Example (contd.)
6 equivalence classes:
E1: x<3, y<5

E3: x<3, y>9
E2: x<3, 5≤y≤9
E4: 3≤x≤7, y<5
9 equivalence classes:
E5: 3≤x≤7, 5≤y≤9
E6: 3≤x≤7, y>9
E7: x>7, y<5
E8: x>7, 5≤y≤9
E9: x>7, y>9
36
Systematic procedure for
equivalence partitioning
1. Identify the input domain: Read the requirements carefully and identify
all input and output variables, their types, and any conditions associated
with their use.
Environment variables, such as class variables used in the method under

test and environment variables in Unix, Windows, or other operating
systems, also serve as input variables.
Given the set of values each variable can assume, an approximation to

the input domain is the cross product of these sets.
37
equivalence partitioning (contd.)
2. Equivalence classing: Partition the set of values of each variable into

disjoint subsets. Each subset is an equivalence class. Together, the
equivalence classes based on an input variable partition the input domain.
Partitioning the input domain using values of one variable is done based on
the expected behavior of the program.
Values for which the program is expected to behave in the “same way” are
grouped together (“same way” needs to be defined by the tester).

38
3. Combine equivalence classes: This step is usually omitted and the

equivalence classes defined for each variable are directly used to select
test cases. However, by not combining the equivalence classes, one
misses the opportunity to generate useful tests.
The equivalence classes are combined using the multidimensional

partitioning approach described earlier.

39
4. Identify infeasible equivalence classes: An infeasible equivalence class is

one that contains a combination of input data that cannot be generated
during test. Such an equivalence class might arise due to several reasons.
For example, suppose that an application is tested via its GUI, i.e., data is
input using commands available in the GUI. The GUI might disallow invalid
inputs by offering a palette of valid inputs only. There might also be
constraints in the requirements that render certain equivalence infeasible.

40
Boiler control example (BCS)
The control software of BCS, abbreviated as CS, is required to offer
several options. One of the options, C (for control), is used by a human
operator to give one of thre commands cmd:
•  temp: change the boiler temperature
•  shut: shut down the boiler
•  cancel: cancel the request
Command temp causes CS to ask the operator to enter the amount by
which the temperature is to be changed (tempch).
Values of tempch are in the range -10..10 in increments of 5 degrees

Fahrenheit.
A temperature change of 0 is not an option.

41
BCS: example (contd.)
Selecting option C forces the BCS to examine variable V.

If V is set to GUI, the operator is asked to enter one of the three
commands via a GUI.
If V is set to file, BCS obtains the command from a command file.
The command file may contain any one of the three commands, together
with the value of the temperature to be changed if the command is temp.
The file name is obtained from variable F.

42
V, F : Environment variables
cmd: command V F
(temp, shut, cancel)
cmd
Control Software
GUI
tempch (CS)
tempch: desired temperature

change (-10..10)
datafile
V ∈ {GUI, file}
F: file name if V is set to “file.”

43
Values of V and F can be altered by a different module in BCS.

In response to temp and shut commands, the control software is required to
generate appropriate signals to be sent to the boiler heating system.
The control software is to be tested in a simulated environment.

The tester takes on the role of an operator and interacts with the BCS via a
GUI.
The GUI forces the tester to select from a limited set of values as specified
in the requirements. For example, the only options available for the value
of tempch are -10, -5, 5, and 10.
These four values of tempch are tvalid, while all other values are tinvalid.

44
BCS: 1. Identify input domain
The first step in generating equivalence partitions is to identify the

(approximate) input domain.
Recall that the domain identified in this step will likely be a superset of the
complete input domain of the control software.
First we examine the requirements, identify input variables, their types,

and values. These are listed in the following table.

45
BCS: Variables, types, values
Variable Kind Type Value(s)
V Environment Enumerated file, GUI
F Environment String A file name
cmd Input via GUI/ Enumerated {temp, cancel, shut}

File
tempch Input via GUI/ Enumerated {-10, -5, 5, 10}

File

46
BCS: Input domain
Input domain ⊆ S = V × F × cmd × tempch
Sample values in the input domain (--: don’t care):
(GUI, --, shut, --), (file, cmdfile, shut, --)
(file, cmdfile, temp, 0) Does this belong to the input domain?

47
BCS: 2. Equivalence classing
Variable Partition
V {{GUI}, {file}, {undefined}}
F {{fvalid}, {finvalid}}
cmd {{temp}, {cancel}, {shut}, {cinvalid}}
tempch {{tvalid}, {tinvalid}}

48
BCS: 3. Combine equivalence
classes (contd.)
Note that tinvalid, tvalid, finvalid, and fvalid denote sets of values.
“undefined” denotes one value.
There is a total of 3×4×2×5=120 equivalence classes.
Sample equivalence class: {(GUI, fvalid, temp, -10)}
Each of the classes listed above represents an infinite number of input

values for the control software.
For example, {(GUI, fvalid, temp, -10)} denotes an infinite set of values
obtained by replacing fvalid by a string that corresponds to the name of
an existing file. Each value is a potential input to the BCS.

49
BCS: 4. Discard infeasible
equivalence classes
The GUI requests for the amount by which the boiler temperature is to be
changed only when the operator selects temp for cmd.
Thus, all equivalence classes that match the following template are infeasible.
{(V, F, {cancel, shut, cinvalid}, tvalid ∪ tinvalid)}
This parent-child relationship between cmd and tempch renders

infeasible a total of 3×2×3×5=90 equivalence classes.
After having discarded all infeasible equivalence classes, we are left

with a total of 18 testable (or feasible) equivalence classes.

50
Selecting test data
Given a set of equivalence classes that form a partition of the input

domain, it is relatively straightforward to select tests.
However, complications could arise in the presence of infeasible data and

don't care values.
In the most general case, a tester simply selects one test that
serves as a representative of each equivalence class.

51
GUI design and equivalence classes
While designing equivalence classes for programs that obtain input

exclusively from a keyboard, one must account for the possibility of errors
in data entry. For example, the requirement for an application.
The application places a constraint on an input variable X such that it

can assume integral values in the range 0..4.
However, testing must account for the possibility that a user may
inadvertently enter a value for X that is out of range.

52
GUI design and equivalence classes
(contd.)
Suppose that all data entry to the application is via a GUI front end. Suppose
also that the GUI offers exactly five correct choices to the user for X.
In such a situation it is impossible to test the application with a value of X
that is out of range. Hence only the correct values of X will be input.

53
Boundary value analysis
54
Errors at the boundaries
Experience indicates that programmers make mistakes in processing values
at and near the boundaries of equivalence classes.
For example, suppose that method M is required to compute a function f1

when x≤0 is true and function f2 otherwise. However, M has an error due
to which it computes f1 for x<0 and f2 otherwise.
Obviously, this fault is revealed, though not necessarily, when M is tested

against x=0 but not if the input test set is, for example, {-4, 7} derived
using equivalence partitioning. In this example, the value x=0, lies at the
boundary of the equivalence classes x≤0 and x>0.

55
Boundary value analysis (BVA)
Boundary value analysis is a test selection technique that targets faults in

applications at the boundaries of equivalence classes.
While equivalence partitioning selects tests from within equivalence

classes, boundary value analysis focuses on tests at and near the
boundaries of equivalence classes.
Certainly, tests derived using either of the two techniques may overlap.

56
BVA: Procedure
1  Partition the input domain using unidimensional partitioning. This leads

to as many partitions as there are input variables. Alternately, a single
partition of an input domain can be created using multidimensional
partitioning. We will generate several sub-domains in this step.
2  Identify the boundaries for each partition. Boundaries may also be

identified using special relationships amongst the inputs.
3  Select test data such that each boundary value occurs in at least
one test input.
57
BVA: Example: 1. Create
equivalence classes
Assuming that an item code must be in the range 99..999 and quantity in
the range 1..100,
Equivalence classes for code:

E1: Values less than 99.
E2: Values in the range.
E3: Values greater than 999.
Equivalence classes for quantity:

E4: Values less than 1.
E5: Values in the range.
E6: Values greater than 100.

58
BVA: Example: 2. Identify boundaries
98 100 998 1000

* x * * x *
E1 99 999 E3
E2
0 2 99 101
* x * * x *
E4 1 100 E6
E5
Equivalence classes and boundaries for findPrice.
Boundaries are marked with an “x”, points near the boundary with a *.
59
BVA: Example: 3. Construct test set
Test selection based on the boundary value analysis technique requires

that tests must include, for each variable, values at and around the
boundary. Consider the following test set:
T={ t1: (code=98, quantity=0),

t2: (code=99, quantity=1), Illegal values of code
t3: (code=100, quantity=2), and quantity included.
t4: (code=998, quantity=99),
t5: (code=999, quantity=100),
t6: (code=1000, quantity=101)
}

60
BVA: Recommendations
Relationships among the input variables must be examined carefully

while identifying boundaries along the input domain.
This may lead to boundaries that are not evident from equivalence
classes obtained from the input and output variables.
Additional tests may be obtained when using a partition of the input

domain obtained by taking the product of equivalence classes created
using individual variables.

61
Testing predicates
62
Where do predicates arise?
Predicates arise from requirements in many applications. Here is an example

from Paradkar, Tai, and Vouk, “Specification based testing using cause-effect
graphs, Annals of Software Engineering,” V. 4, pp 133-157, 1997.
A boiler needs to be to be shut down when the following conditions hold:
1.  The water level in the boiler is below X lbs. (a)
2.  The water level in the boiler is above Y lbs. (b)
3.  A water pump has failed. (c) Boiler is in degraded mode
4.  A pump monitor has failed. (d) when either is true.
5.  Steam meter has failed. (e)
The boiler is to be shut down when a or b is true or the boiler is in degraded
mode and the steam meter fails. We combine these five conditions to form a
compound condition (predicate) for boiler shutdown.
63
Boiler shutdown conditions
Denoting the five conditions above as a through e, we obtain the

following Boolean expression E that, when true, must force a boiler
shutdown:
E = a+b+(c+d)e
where the + sign indicates “OR” and a multiplication indicates “AND.”
The goal of predicate-based test generation is to generate tests from a

predicate p that guarantee the detection of any error that belongs to a
class of errors in the coding of p.

64
Another example
A condition is represented formally as a predicate, or Boolean expression.

For example, consider the requirement
“if the printer is ON and has paper then send document to printer.”
This statement consists of a condition part and an action part. The

following predicate represents the condition part of the statement.
pr: (printerstatus=ON) AND (printertray!= empty)

65
Test generation from predicates
We now examine two techniques, named BOR and BRO, for generating
tests guaranteed to detect certain faults in the coding of conditions.
The conditions from which tests are generated might arise from
requirements or might be embedded in the program to be tested.
Conditions guard actions. For example,

if condition then action
This is a typical format of many functional requirements.

66
Predicates
Relational operators (relop): {<, <=, >, >=, =, !=}
Boolean operators (bop): {NOT, AND, OR, XOR}.
Relational expression: e1 relop e2. (e.g. a+b<c)

e1 and e2 are expressions whose values can be compared using relop.
Simple predicate: A Boolean variable or a relational expression.

(x<0)
Compound predicate: Join one or more simple predicates using bop.

(gender=“female” OR age>65)

67
Boolean expressions
Boolean expression: one or more Boolean variables joined by bop.

(a AND b OR NOT c)
a, b, and c are literals.

Negation is also denoted by placing a bar over a Boolean expression.
We also write ab for a AND b and a+b for a OR b when there is no confusion.
Singular Boolean expression: When each literal appears only once,

(a AND b OR NOT c)

68
Boolean expressions (contd.)
Disjunctive normal form (DNF): Sum of products:
e.g., (p q) +(r s) + (a c).
Conjunctive normal form (CNF): Product of sums:

e.g., (p+q)(r+s)(a+c)
Any Boolean expression in DNF can be converted to an equivalent CNF

and vice versa.
e.g., CNF: (p+!r)(p+s)(q+!r)(q+s) is equivalent to DNF: (p q + !r s)
Boolean expressions e1 and e2 are mutually singular if they do not share

literals. If expression E contains components e1, e2,.. then ei is considered
singular only if it is singular and mutually singular with every other
component ej of E.
69
Boolean expressions: Syntax tree
representation
Abstract syntax tree (AST) for:

(a+b)<c AND NOT p. Root node
Internal nodes are labeled by
boolean and relational operators AND
< NOT
(a+b) c p
Leaf nodes
70
Fault model for predicate testing
What faults are we targeting when testing for the

correct implementation of predicates?
Boolean operator fault:

Suppose that the specification of a software module requires that an
action be performed when the condition (a<b) OR (c>d) AND e is true.
Here a, b, c, and d are integer variables and e is a Boolean variable.

71
Boolean operator faults
Correct predicate: (a<b) OR (c>d) AND e
(a<b) AND (c>d) AND e Incorrect Boolean operator

(a<b) OR NOT (c>d) AND e Incorrect negation operator
(a<b) AND (c>d) OR e Incorrect Boolean operators
(a<b) OR (e>d) AND c Incorrect Boolean variable.

72
Relational operator faults
Correct predicate: (a<b) OR (c>d) AND e
(a=b) OR (c>d) AND e Incorrect relational operator

(a=b) OR (c<=d) AND e Two relational operator faults
(a=b) OR (c>d) OR e Incorrect Boolean operators

73
Arithmetic expression faults
Correct predicate: Ec: e1 relop1 e2.

Incorrect predicate: Ei: e3 relop2 e4.
Assume that Ec and Ei use the same set of variables.
Ei has an off-by-ε fault if |e3-e4|= ε for any test case for which e1=e2.
Ei has an off-by-ε* fault if |e3-e4|>= ε for any test case for which e1=e2.
Ei has an off-by-ε+ fault if |e3-e4|> ε for any test case for which e1=e2.

74
Arithmetic expression faults:
Examples
Correct predicate: Ec: a<(b+c). Assume ε=1.
Ei: a<b. Given c=1, Ei has an off-by-1 fault as |a-b|=1 for a test case
for which a=b+c, e.g., [a=2, b=1, c=1].
Ei: a<b+1. Given c=2, Ei has an off-by-1* fault as |a-(b+1)|>= 1 for

any test case for which a=b+c, e.g., [a=4, b=2, c=2].
Ei: a<b-1. Given c>0, Ei has an off-by-1+ fault as |a-(b-1)|>1 for any
test case for which a=b+c, e.g., [a=3, b=2, c=1].
75
Goal of predicate testing
Given a correct predicate pc , the goal of predicate testing is to generate a
test set T such that there is at least one test case t in T for which pc and its
faulty version pi evaluate to different truth values.
Such a test set guarantees the detection of any fault of the kind in the
fault model introduced above.
As an example, suppose that pc: a<b+c and pi: a>b+c.

Consider T={t1, t2} with t1: [a=0, b=0, c=0] , t2: [a=0, b=1, c=1].
The fault in pi is not revealed by t1 as both pc and pi evaluate to false when

evaluated against t1.
The fault is revealed by t2 as pc evaluates to true and pi to false when
evaluated against t2.
76
Missing or extra Boolean variable
faults
Correct predicate: a OR b
Missing Boolean variable fault: a
Extra Boolean variable fault: a OR b AND c

77
Predicate constraints: BR symbols
Consider the following Boolean-Relational set of BR-symbols:

BR={t, f, <, =, >, +ε, -ε}
A BR symbol is a constraint on a Boolean variable or a relational

expression.
For example, consider the predicate E: a”.
A test case that satisfies this constraint for E must cause E to

evaluate to false.
78
Infeasible constraints
A constraint C is considered infeasible for predicate pr if there exists

no input values for the variables in pr that satisfy c.
For example, the constraint t is infeasible for the predicate

a>b AND b>d
if it is known that d>a.

79
Predicate constraints
Let pr denote a predicate with n>0 OR or AND operators.
A predicate constraint C for predicate pr is a sequence of (n+1) BR

symbols, one for each Boolean variable or relational expression in pr.
When clear from the context, we refer to “predicate constraint” as
simply constraint.
Test case t satisfies C for predicate pr , if each component of pr

satisfies the corresponding constraint in C when evaluated against t.
Constraint C for predicate pr guides the development of a test for pr ,
i.e., it offers hints on what the values of the variables should be for pr
to satisfy C.
80
True and false constraints
pr(C) denotes the value of predicate pr evaluated using a test case
that satisfies C.
C is referred to as a true constraint when pr(C) is true and a false
constraint otherwise.
A set of constraints S is partitioned into subsets St and Sf, such that,
for each C in St, pr(C )=true, and for any C in Sf, pr(C ) =false.
Consider predicate pr: b AND (r<s) OR (u ≥v) and constraint C : (t, =, >).
The following test case satisfies C for pr.
[b=true, r=1, s=1, u=1, v=0]
The following test case does not satisfy C for pr.
[b=true, r=1, s=2, u=1, v=2]
81
Predicate testing: criteria
Given a predicate pr , we want to generate a test set T such that
•  T is minimal and
•  T guarantees the detection of any fault in the implementation of pr
(faults correspond to the fault model we discussed earlier).
We will present three such criteria: BOR, BRO, and BRE.

82
Predicate testing: BOR, BRO, and
BRE testing criterion
A BOR-adequate test set TBOR satisfying the BOR testing criterion for a
compound predicate pr guarantees the detection of single or
multiple Boolean operator faults in the implementation of pr .
A BRO-adequate test set TBRO satisfying the BRO testing criterion for a
compound predicate pr, guarantees the detection of single or multiple
Boolean operator and relational operator faults in the
implementation of pr .
A BRE-adequate test set TBRE satisfying the BRE testing criterion for a
compound predicate pr guarantees the detection of single or
multiple Boolean operator, relational expression, and
arithmetic expression faults in the implementation of pr .
83
Predicate testing: guaranteeing fault
detection
Let Tx, x ∈ {BOR, BRO,BRE}, be a test set derived from predicate pr.
Let pf be another predicate obtained from pr by injecting single or multiple
faults of one of three kinds: Boolean operator fault, relational operator fault,
and arithmetic expression fault.
Tx detects faults in pf if, for some t ∈Tx , pr (t)≠ pf (t).
Example: let pr=ad. Constraint set S={(t, t), (t,f ), (f, t)}
TBOR={t1, t2, t3} is a BOR adequate test set that satisfies S.
t1: [a=1, b=2, c=1, d=0] Satisfies (t, t), i.e. a<b is true and c<d is true.
t2: [a=1, b=2, c=1, d=2] Satisfies (t, f )
t3: [a=1, b=0, c=1, d=0] Satisfies (f, t)
84
Set products: Example
Recall that the cross product A×B of two sets A and B is defined as:
A×B={(u,v)|u ∈ A and v ∈ B }
An onto product A⊗B of two sets A and B is defined as any minimal

subset of A×B s.t., for each element of u ∈ A, there is a pair (u,v) in
A⊗B and, for each element of v ∈ B, there is a pair (u,v) in A⊗B.
Let A={t, =, >} and B={f, <}
A×B={(t, f), (t, <), (=, f), (=, <), (>,f), (>,<)}
A⊗B ={(t, f), (=,<), (>,<)} Any other possibilities for A⊗B?

85
Generation of BOR constraint set
See page 160 for a formal algorithm.

AND
We want to generate TBOR for pr : ad
First, generate the syntax tree of pr.

ad
We use the following notation:

SN is the constraint set for node N in the syntax tree for pr .
StN is the true constraint set for node N in the syntax tree for pr .
SfN is the false constraint set for node N in the syntax tree for pr .
SN = St N ∪ Sf N .
86
Generation of the BOR constraint set
(contd.)
Second, label each leaf node with the constraint set {(t),(f)}.
We label the nodes as N1, N2, and so on for convenience.
N1 and N2 are direct descendents of N3 which is an AND-node.
N3 AND
N1 ad N2
SN1= {(t),(f)} SN2= {(t),(f)}

87
Generation of the BOR constraint set
(contd.)
Third, compute the constraint set for the next higher node in the syntax tree,
in this case N3. For an AND node, the formulae used are the following.
SN3t = StN1 ⊗ StN2 ={(t)} ⊗ {(t)}={(t,t)} SN3={(t,t),(f,t),(t,f)}

N3 AND
SfN3 = (Sf N1×{tN2})∪({tN1}× SfN2 )
= ({(f)} ×{(t)})∪({(t)}× {(f)}) N1 N2

ad
= {(f,t)}∪{(t,f)} {(t),(f)} {(t),(f)}
= {(f,t),{(t,f)}
Note: tN denotes an element of StN. fN denotes an element of SfN

88
Generation of TBOR
As per our objective, we have computed the BOR constraint set for the
root node of the AST(pr). We can now generate a test set using the
BOR constraint set associated with the root node.
SN3 contains a sequence of three constraints,

SN3={(t,t), (f,t), (t,f)}
thus we get a minimal test set of three test
cases. Here is one possible test set. N3 AND
TBOR ={t1, t2, t3} N1 N2

T1=[a=1, b=2, c=6, d=5] (t, t) ad
T2=[a=1, b=0, c=6, d=5] (f, t) {(t),(f)} {(t),(f)}
T3=[a=1, b=2, c=1, d=2] (t, f)
89
Generation of BRO constraint set
See page 137 for a formal algorithm.

Recall that a test set adequate with respect to a BRO constraint set
for predicate pr, guarantees the detection of all combinations of
single or multiple Boolean operator and relational operator faults.
The BRO constraint set S for e1 relop e2 is S={(>), (=), (<)}

Separation of S into its true (St ) and false (Sf ) components:
relop: > St={(>)} Sf={(=), (<)}
relop: ≥ St={(>), (=)} Sf={(<)}
relop: = St={(=)} Sf={(<), (>)}
relop: < St={(<)} Sf={(=), (>)}
relop: ≤ St={(<), (=)} Sf={(>)}

90
BRO constraint set: Example
pr : (a+b<c) AND NOT p OR (r >s )
Step 1: Construct the AST for the given predicate.

N6
OR
N4
N5
AND
r >s
N1 N3
a+b<c NOT
p N2
91
BRO constraint set: Example (contd.)
Step 2: Label each leaf node with its constraint set S.
OR N6
N4
AND N5
r >s
N1 N3 {(>), (=), (<)}
a+b<c NOT
{(>), (=), (<)}
p N2
{(t), (f)}
92
Step 2: Traverse the tree and compute constraint set for each internal node.
StN3=SfN2={(f)} SfN3=StN2= {(t)}
StN4=StN1 ⊗ StN3={(<)} ⊗ {(f)}={(<, f)}
SfN4= (SfN1 × {tN3}) ∪ ({tN1} × SfN3)

=({(>,=)} × {(f)}) ∪ {(<)} × {(t)})
={(>, f), (=, f)} ∪ {(<, t)}
={(>, f), (=, f), (<, t)}

93
N6
OR
N4
AND N5
{(<,f),(>,f),(=,f),(<,t)}
r >s
{(>), (=), (<)}
N3 {(f), {t)}
N1 a+b<c NOT
{(>), (=), (<)}
p N2
{(t), (f)}

94
Next, compute the constraint set for the root node (an OR-node).
SfN6=SfN4 ⊗ SfN5
={(>,f), (=,f), (<,t)} ⊗ {(=), (<)}
={(>,f,=), (=,f,<), (<,t,=)}
StN6= (StN4 × {fN5} ) ∪ ({fN4} × StN5)

=({(<,f)} × {(=)}) ∪ {(>,f)} × {(>)})
={(<,f,=)} ∪ {(>,f,>)}
={(<,f,=), (>,f,>)}

95
Constraint set for Pr : (a+b<c) AND NOT p OR (r >s)
{(>,f,=), (=,f,<),(<,t,=), (<,f,=),(>,f,>)}

OR N6
N4 N5
{(<,f),(>,f),(=,f),(<,t)} AND
r >s
{(>),(=),(<)}
N1 N3 {(f), {t)}
a+b <c NOT
{(>),(=),(<)}
p N2
{(t),(f)}
96
BRO constraint set: In-class exercise
Given the constraint set for pr : (a+b<c) AND NOT p OR (r >s), build TBRO.
{(>,f,=), (=,f,<),(<,t,=), (<,f,=),(>,f,>)}
Reading assignment:
Section 2.7.5.3, “Generating the BRE constraint set”, pages 165-167.

97
BOR constraints for non-singular
expressions
Test generation procedures described so far work for singular predicates.

Recall that a singular predicate contains only one occurrence of each variable.
We now learn how to generate BOR constraints for non-singular predicates.
First, let us look at some non-singular expressions, their respective

disjunctive normal forms (DNF), and their mutually singular components.

98
Non-singular expressions and DNF:
Examples
Predicate (pr) DNF Mutually singular

components in pr
ab(b+c) abb+abc a; b(b+c)
a(bc+bd) abc+abd a; bc+bd
a(bc+!b+de) abc+a!b+ade a; bc+!b; de

99
Generating BOR constraints for non-
singular expressions
We proceed in two steps.
First, we examine the Meaning Impact (MI) procedure to generate a

minimal set of constraints from a possibly non-singular predicate.
Next, we examine the procedure to generate BOR constraint set for a

non-singular predicate.

100
Meaning Impact (MI) procedure
Given boolean expression E in DNF, the MI procedure generates a set

of constraints SE that guarantees the detection of missing or extra
NOT (!) operator faults in the implementation of E.
The MI procedure is on pages 168-169 of the textbook.

We illustrate it with an example.
Consider the non-singular predicate: a(bc+!bd). Its DNF equivalent is:

E=abc+a!bd.
Note that a, b, c, and d are boolean variables, also referred to as literals.
Each literal represents a condition. For example, a could represent r <s.
101
MI procedure: Example (contd.)
Step 0: Express E in DNF notation. Clearly, we can write E=e1+e2,

where e1=abc and e2=a!bd.
Step 1: Build a constraint set Te1 for e1 that makes e1 true and a
constraint set Te2 for e2 that makes e2 true.
Te1 ={(t,t,t,t), (t,t,t,f)} Te2 ={(t,f,t,t), (t,f,f,t)}
The four t’s in the first element of Te1 denote the values of the boolean
variables a, b, c, and d, respectively. The second element, and others, are
to be interpreted similarly.

102
Step 2: From each Tei , remove the constraints that are in any other Tej.
This gives us TSei and Tsej , where TSei ∩TSej =∅.
There are no common constraints between Te1 and Te2 in our example.
Hence we get:
TSe1 ={(t,t,t,t), (t,t,t,f)} TSe2 ={(t,f,t,t), (t,f,f,t)}
Step 3: Build StE by selecting one element from each TSei.

StE ={(t,t,t,t), (t,f,f,t)}
For each constraint x in StE we get E(x)=true. Also, StE is minimal. Check it out!

103
Step 4: For each term in E, obtain terms by complementing each

literal, one at a time.
e11= !abc e21= a!bc e31= ab!c
e12= !a!bd e22= abd e32= a!b!d

From each term e above, derive constraints Fe that make e true.
We obtain the following six sets:
Fe11= {(f,t,t,t), (f,t,t,f)} Fe12= {(f,f,t,t), (f,f,f,t)}

Fe21= {(t,f,t,t), (t,f,t,f)} Fe22= {(t,t,t,t), (t,t,f,t)}
Fe31= {(t,t,f,t), (t,t,f,f)} Fe32= {(t,f,t,f), (t,f,f,f)}
104
Step 5: Build FSe by removing from Fe any constraint that appeared in
any of the two sets Te built earlier.
FSe11= Fe11 = {(f,t,t,t), (f,t,t,f)}

FSe21= {(t,f,t,f)}
FSe31= Fe31= {(t,t,f,t), (t,t,f,f)} Constraints common with
Te1 and Te2 are removed.
FSe12= Fe12= {(f,f,t,t), (f,f,f,t)}

FSe22= {(t,t,f,t)}
FSe32= Fe32= {(t,f,t,f), (t,f,f,f)}

105
Step 6: Build SfE by selecting one constraint from each FSe.
SfE ={(f,t,t,f), (t,f,t,f), (t,t,f,t), (f,f,t,t)}
Step 7: Build SE= StE ∪SfE
SE={{(t,t,t,t), (t,f,f,t), (f,t,t,f), (t,f,t,f), (t,t,f,t), (f,f,t,t)}
Note: Each constraint in StE makes E true and each constraint in SfE
makes E false.
We are now done with the MI procedure.
106
BOR-MI-CSET procedure
The BOR-MI-CSET procedure takes a non-singular expression E as

input and generates a constraint set that guarantees the detection of
Boolean operator faults in the implementation of E.
The BOR-MI-CSET procedure using the MI procedure described earlier.
The entire procedure is described on page 171.
We illustrate it with an example.

107
BOR-MI-CSET: Example
Consider a non-singular Boolean expression: E= a(bc+!bd)
Mutually singular components of E : e1=a e2=bc+!bd
We use the BOR-CSET procedure to generate the constraint set for e1 (singular
component) and MI-CSET procedure for e2 (non-singular component).
For component e1 we obtain:

Ste1={t} Sfe1={f}
(Ste1 is the true constraint set for e1 and Sfe1 is the false constraint set for e1).

108
BOR-MI-CSET: Example (contd.)
Component e2 is a DNF expression, e2=u+v where u =bc and v =!bd.

Apply the MI-CSET procedure to obtain the BOR constraint set for e2.
Applying Step 1 of the MI-CSET procedure we obtain:
Tu={(t,t,t), (t,t,f)} Tv={(f,t,t), (f,f,t)}
Applying Steps 2 and 3 to Tu and Tv we obtain:
TSu=Tu TSv=Tv One possible alternative.
Ste2={(t,t,f), (f,t,t)} Can you think of other alternatives?

109
Next we apply Step 4 to u and v.

We obtain the following complemented expressions from u and v :
u1=!bc u2=b!c
v1=bd v2=!b!d
Continuing with Step 4 we obtain:
Fu1={(f,t,t), (f,t,f)} Fu2={(t,f,t), (t,f,f)}

Fv1={(t,t,t), (t,f,t)} Fv2={(f,t,f), (f,f,f)}

110
Next we apply Step 5 to the F constraint sets to obtain:

FSu1={(f,t,f)} FSu2={(t,f,t), (t,f,f)}
FSv1={(t,f,t)} FSv2={(f,t,f), (f,f,f)}
Applying Step 6 to the FS sets leads to:

Sfe2={(f,t,f), (t,f,t)}
Combing the true and false constraint sets for e2 we obtain:

Se2={(t,t,f), (f,t,t), (f,t,f), (t,f,t)}
111
Summary:
Ste1={(t)} Sfe1={(f)} from the BOR-CSET

procedure.
Ste2={(t,t,f), (f,t,t)} Sfe2={(f,t,f), (t,f,t)} from the MI-CSET

procedure.
We now apply Step 2 of the BOR-CSET procedure to obtain

the constraint set for the entire expression E.

112
Obtained by applying Step 2 of BOR-CSET
StN3=StN1 ⊗ StN2 to an AND node.
SfN3=(SfN1 × {tN2})∪({tN1} × SfN2) N3

AND {(t,t,t,f),(t,f,t,t),(f,t,t,f),(t,f,t,f),(t,t,f,t)}
N2
OR {(t,t,f), (f,t,t), (f,t,f), (t,f,t)}
AND AND
N1
a {(t),(f)}
b c !b d
Apply MI-CSET
113
Summary
Equivalence partitioning and boundary value analysis are the most commonly
used methods for test generation while doing functional testing.
Given a function f to be tested in an application, one can apply these

techniques to generate tests for f.
Most requirements contain conditions under which functions are to be

executed. The predicate testing procedures covered are excellent means to
generate tests to ensure that each condition is tested adequately.

114
Summary (contd.)
Usually one would combine equivalence partitioning, boundary

value analysis, and predicate testing procedures to generate tests
for a requirement of the following type:
if condition then action 1, action 2, …action n;
Apply predicate testing

Apply equivalence partitioning, BVA, and
predicate testing if there are nested conditions.

115

Chapter 2

Diunggah oleh

Informasi Dokumen

Deskripsi Asli:

Hak Cipta

Format Tersedia

Bagikan dokumen Ini

Bagikan atau Tanam Dokumen

Opsi Berbagi

Apakah menurut Anda dokumen ini bermanfaat?

Apakah konten ini tidak pantas?

Hak Cipta:

Format Tersedia

Chapter 2

Diunggah oleh

Hak Cipta:

Format Tersedia

Foundations of Software Testing

Chapter 2: Test Generation: Requirements

These slides are copyrighted. They are for use

Last update: October 6, 2010

 Equivalence class partitioning

 Test generation from predicates

© Aditya P. Mathur 2006

Test generation techniques described in this chapter belong to the

© Aditya P. Mathur 2006

Test Plan Test Design Test Case Test Procedure

Test item transmittal Test log Test incident

Test design spec: Identifies a subset of features to be tested and

Test case spec: Lists inputs, expected outputs, features to be

Test log: A log of the observations during the execution of a test.

Test incident report: Document any special event recommended for

Test summary: Summarizes the results of testing activities and

Each of these test generation techniques is a black-box technique

© Aditya P. Mathur 2006

Requirements serve as the starting point for the generation of tests.

These requirements, more aptly ideas, are then specified rigorously

Rigorously specified requirements are often transformed into formal

© Aditya P. Mathur 2006

© Aditya P. Mathur 2006

In general, there is no algorithm to build such a test set.

However, there are heuristics and model-based methods that can be

The challenge is to build a test set T ⊆ D that will reveal as many

By “exhaustive” testing we mean testing the given program against

© Aditya P. Mathur 2006

Assuming that P will be executed on a machine where integers range

If there is no limit on the length of the sequence that can be input,

Calculate the size of the input domain.

For simplicity, assume that the employee record consists of the

Calculate the size of the input domain.

© Aditya P. Mathur 2006

The equivalence classes are created assuming that the program

© Aditya P. Mathur 2006

Equivalence class partitioning selects tests that

© Aditya P. Mathur 2006

© Aditya P. Mathur 2006

© Aditya P. Mathur 2006

The effectiveness of tests generated using equivalence partitioning for

The effectiveness can be improved through an unambiguous and complete

© Aditya P. Mathur 2006

This example shows a few ways to define equivalence classes based on

Consider a wordCount method which takes a word w and a filename f as

An exception is raised if there is no file with name f.

© Aditya P. Mathur 2006

E1 non-null exists, not empty

E2 non-null does not exist

E3 non-null exists, empty

E4 null exists, not empty

E5 null does not exist

E6 null exists, empty

© Aditya P. Mathur 2006

The number of equivalence classes without any knowledge of

Of course, an experienced tester will likely derive the six equivalence

© Aditya P. Mathur 2006

These questions lead to two the following equivalence classes:

Equivalence Classes Examples

area: float {{-1.0}, {15.52}}

letter: bool {{J}, {3}}

© Aditya P. Mathur 2006

Equivalence Classes Example

  Equivalence class partitioning

  Test generation from predicates