Software Testing Techniques

How to find bugs in the software?

Validation & Verification

Validation
Are we building the product right? Ensure software meets customers needs

Verification
Are we building the right product? Ensure software meet specification (errorfree)
2

Testing Terminology
Failure
Incorrect or unexpected output Symptom of at fault

Fault

Invalid execution state Symptom of an error May or may not produce failure

Error

Defect or anomaly in source code Commonly referred as bug May or may not produce fault
3

Testing Objective
To design tests that systematically uncover different classes of errors and to do so with a minimum amount of time

and effort

Why We Need Software Testing?

Reveal faults/failures/errors Locate faults/failures/errors Show system correctness Improved confidence that system performs as specified (verification) Improved confidence that system performs as desired (validation) Indicator of system reliability and system quality
5

Importance of Testing
Critical Element of Software Quality Assurance Well-planned and thorough testing are important in reducing the high cost of software failure Can take up to 40% of development effort In systems that require high reliability, testing can be 3 to 5 times of all the other 6 steps combined

Testing Principles
All test should be traceable to customer requirements. Tests should be planned before testing begins. Testing should begin with individual components and move towards to integrated cluster of components. The most effective testing should be conducted by an independent third party.
7

Pre-requisites For Good Testing

Adequate staff Adequate support facilities (tools) Adequate time

Who Should Test The Software?

Developer (individual units) Independent test group (ITG)
removes conflict of interest reports to SQA team

Successful Testing
Uncover errors in the software Demonstrates that the software functions appear to be working according to specification Data collected as testing is conducted provide a good indication of software reliability and software quality
10

Testing Guidelines
Testing a systems capabilities is more important than testing its components Testing old capabilities is more important than testing new capabilities Testing typical situations is more important than testing boundary value cases
11

Test Cases
Test case : unit of testing activity Test cases have 3 parts :Goal
Aspect of the system being tested

Input and system state

Data provided to the system under stated condition

Expected behavior
The output or action the system should take according to its requirements

12

Test Cases (Cont)

Test cases are derived during all phase of development cycle Determine the expected result before running the test case Record test cases and its results Allocate
money people resources
13

time

Type Of Test Cases

Test cases are derived for
Valid and expected input Invalid and unexpected input Test if the system does less than specified requirement Test if the system does more than specified requirement

14

Flow Graph Notation

Edge
1 2,3 2 6 3 6 7
9 11 10
R2

Node

4,5
R1

4 8 5

R3

Region 9

10
R4

11

15

Cyclomatic Complexity
Provide quantitative measure for program logical complexity. Defined number of independent path
Any path that introduce one ser of processing statements or new condition

Eg :Path Path Path Path 1 2 3 4 : : : : 1-11 1-2-3-4-5-10-1-11 1-2-3-6-8-9-10-1-11 1-2-3-6-7-9-10-1-11

16

How Is Cyclomatic Complexity Computed?

1. Number of regions
The flow graph has 4 regions E : Number of flow graph edges N : Number of flow graph nodes
V(G) = 11 edges 9 nodes + 2 = 4

2. V(G) = E N + 2

3. V(G) = P + 1
P : Number of predicate nodes
V(G) = 3 predicate nodes + 1 = 4
17

Deriving Test Cases

1. Draw flow graph from design/code as foundation.
i=1; 2 total.input = total.valid=0; 1 sum=0; do while value[i] <> -999 and total.input<100 3 4 increment total.input by 1; if value[i] >= minimum AND value[i] <= maximum 6 5 then increment total.valid by 1; sum = sum + value[i] 7 else skip end if 8 increment i by 1 9 End do If total.valid > 0 10 11 then average = sum / total valid; 12 else average = -999; 13 End if

1 2 3 10 12 13 8 9 18 11 4 5

6
7

Deriving Test Cases (cont)

2. Determine cyclomatic complexity
V(G) = 6 regions V(G) = 17 edges 13 nodes + 2 = 6 V(G) = 5 predicates nodes + 1 = 6 Path 1 = 1-2-10-11-13 Path 2 = 1-2-10-12-13 . Path 1 test case :
value(k) = valid input where k < i defined below value(i) = -999 where 2 <= i <= 100 Expected result : Correct average based on k value and proper totals
19

3.

Determine a basis set of linearly independent graph

4.

Prepare test cases

Discussion on White Box Testing

Advantages
Find errors on code level Typically based on a very systematic approach, covering the complete internal module structure

Constraints
Does not find missing or additional functionality Does not really check the interface Difficult for large and complex module

20

TCS2411 Software Engineering

21

Control Structure Testing

Condition testing is a test case design method that exercises the logical conditions contained in a program module. Simple condition is a Boolean variable of a relational expression, E <relational operator>E2. If a condition is incorrect ,

22

Data flow testing selects test paths of a program according to the locations of definitions and uses of variables in the program

TCS2411 Software Engineering

23

TCS2411 Software Engineering

24

Loop Testing

Simple loop Nested Loops Concatenated Loops Unstructured Loops

25

Loop Testing: Simple Loops

Minimum conditionsSimple Loops 1. skip the loop entirely 2. only one pass through the loop 3. two passes through the loop 4. m passes through the loop m < n 5. (n-1), n, and (n+1) passes through the loop where n is the maximum number of allowable passes

26

Loop Testing: Nested Loops

Nested Loops Start at the innermost loop. Set all outer loops to their minimum iteration parameter values. Test the min+1, typical, max-1 and max for the innermost loop, while holding the outer loops at their minimum values. Move out one loop and set it up as in step 2, holding all other loops at typical values. Continue this step until the outermost loop has been tested. Concatenated Loops If the loops are independent of one another then treat each as a simple loop else* treat as nested loops endif* for example, the final loop counter value of loop 1 is used to initialize loop 2.
27

Black Box Testing

Derived from program specification Functional testing of a component of a system Examine behaviour through inputs & the corresponding outputs Input is properly accepted, output is correctly produced Disregard internal logical structure
28

Black Box Testing

(Continued)

Attempts to find the following errors: Incorrect or missing functions Interface errors Errors in data structures or external database access Performance errors Initialisation and termination errors
29

Black Box Testing Techniques

TCS2411 Software Engineering

30

Equivalence Partitioning
Divide input domain into classes of data Based on an evaluation of equivalence classes for an input condition Guidelines to define equivalence classes
Range input : One valid and two invalid equivalence Specific value : One valid and two invalid equivalence A member of a set : One valid and one invalid equivalence Boolean : One valid and one invalid equivalence
31

Example Data for Automated Banking Application

The use can access the bank using his personal computer, provide a six digit password, and follow with a series of typed commands that trigger various banking function. During the log on sequence the software supplied for the banking application accepts data in the form: area code blank or 3 digit numbers prefix 3 digit numbers, nor beginning with 0 or 1 suffix 4 digit numbers password 6 digits alphanumeric strings commands check, deposit, bill pay etc Input condition
area code : Input condition : Boolean area code may or may not present Input condition : Range 200 999 with specific exception prefix : Input condition : Range specified value > 200 with no 0 digits suffix : Input condition : Value 4 digit length password : Input condition : Boolean password may or may not present Input condition : Value six character string command : Input condition : Set containing command 32

Boundary Value Analysis

Complement equivalence partitioning Test both sides of each boundary Look at output boundaries for test cases Test min, min-1, max, max+1, typical values Example : 1 <= x <=100
Valid : 1, 2, 99, 100 Invalid : 0 and 101
33

Discussion on Black Box Testing

Advantages
Find missing functionality Independent from code size and functionality Find some coding errors

Constraints
No systematic search for low level errors Specification errors not found
34

References
Software Engineering: A Practitioners Approach 5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001 Software Engineering by Ian Sommerville, Addison-Wesley, 2001

35