SOFTWARE TESTING

Objectives of testing

Executing a program with the intent of finding an error. To check if the system meets the requirements and be executed successfully in the Intended environment. To check if the system is Fit for purpose. To check if the system does what it is expected to do.

Software Testing Life Cycle

Requirements Analysis
Identify types of tests to be performed. Gather details about testing priorities and focus. Prepare Requirement Traceability Matrix (RTM). Identify test environment details where testing is supposed to be carried out.

Test Planning
Preparation of test plan/strategy document for various types of testing. Test tool selection Test effort estimation Resource planning and determining roles and responsibilities. Training requirement

Test Case Development

Activities Create test cases, automation scripts (if applicable) Review and baseline test cases and scripts Create test data (If Test Environment is available) Deliverables Test cases/scripts Test data

Environment Setup
Understand the required architecture, environment set-up and prepare hardware and software requirement list for the Test Environment. Setup test Environment and test data. can be done in parallel with Test Case Development Stage. Test team may not be involved in this activity.

Test Execution
Execute tests as per plan Document test results, and log defects for failed cases. Map defects to test cases in RTM Retest the defect fixes Track the defects to closure Bugs will be reported back to the development team for correction.

Test Cycle Closure

Evaluate cycle completion criteria based on Time, Test coverage, Cost, Software, Critical Business Objectives, Quality. Prepare test metrics based on the above parameters. Document the learning out of the project. Prepare Test closure report Test result analysis to find out the defect distribution by type and severity.

Testing Methods
Black Box Testing White Box Testing Gray Box Testing

Black Box Testing

The tester interacts with the systems user interface by providing inputs and examining outputs without knowing how and where the inputs are worked upon. Inefficient testing, the tester only has limited knowledge about an application.

Black Box Testing-Steps

Initially requirements and specifications of the system are examined. Tester chooses valid inputs to check whether system under test processes them correctly. Tester determines expected outputs for all the inputs. Tester constructs test cases with the selected inputs which are executed. Tester compares the actual outputs with the expected outputs.

Types of Black Box Testing

Functional testing related to functional requirements of a system; it is done by software testers. Non-functional testing not related to testing of a specific functionality, but non-functional requirements such as performance, scalability, usability. Regression testing done after code fixes, upgrades or any other system maintenance to check the new code has not affected the existing code.

Black Box Testing Strategy

Equivalence Class Testing: used to minimize the number of possible test cases to an optimum level while maintaining reasonable test coverage. Boundary Value Testing: focused on the values at boundaries, determines whether a certain range of values are acceptable by the system or not. It is very useful in reducing the number of test cases. Decision Table Testing: A decision table puts causes and their effects in a matrix. There is unique combination in each column.

White Box Testing

The detailed investigation of internal logic and structure of the code. The tester needs to possess knowledge of the internal working of the code. Use of specialized tools like code analyzers and debugging tools are required. Extra lines of code can be removed which can bring in hidden defects.

What do you verify in White Box Testing

Internal security holes Broken or poorly structured paths in the coding processes The flow of specific inputs through the code Expected output The functionality of conditional loops Testing of each statement, object and function on an individual basis.

How to perform White Box Testing

Understand the source code : involves the testing of the inner workings of an application, the testing person must be highly aware of secure coding practices. Create test cases and execute : involves testing the applications source code for proper flow and structure, requires that the tester must have intimate knowledge of the code and is often done by the developer.

White Box Testing Types

Unit Testing : performed on each unit or block of code as it is developed. Testing for Memory Leaks : essential in cases where you have a slow running software application. White Box Penetration Testing : tester has the applications source code, detailed network information, IP addresses involved and all server information the application runs on. White Box Mutation Testing : used to discover the best coding techniques to use for expanding a software solution.

Gray Box Testing

The technique to test the application with limited knowledge of the internal workings of an application. The tester has access to design documents and the database. Grey box testers dont rely on the source code; instead they rely on interface definition and functional specifications.

Levels of Testing

Unit Testing (White Box) Integration Testing Function Testing (Black Box) Regression Testing System Test User Acceptance Test

Unit Testing

Individual components are tested. It is a path test. To focus on a relatively small segment of code and aim to exercise a high percentage of the internal path. Disadvantage: the tester may be biased by previous experience. And the test value may not cover all possible values.

Integration Testing
Integration Test case differs from other test
cases in the sense that it focuses mainly on the interfaces & flow of data/information between the modules. Here priority is to be given for the integrating links rather than the unit functions which are already tested. Types of Integration Testing: Top-down Integration Test Bottom-up Integration Test

Top-down Integration Test

The control program is tested first. Modules are integrated one at a time. Emphasize on interface testing Advantages: No test drivers needed. Interface errors are discovered early. Modular features aid debugging Disadvantages: Test stubs are needed. Errors in critical modules at low levels are found late.

Top-down Testing
T1 A T2 B B A

T1

T2

T3

T3 C T4

Bottom-up Integration Test

Allows early testing aimed at proving feasibility Emphasizes on module functionality and performance Advantages: No test stubs are needed. Errors in critical modules are found early Disadvantages: Test drivers are needed. Interface errors are discovered late.

Bottom-up testing
Test Drivers Test Drivers Test Drivers

Level N
Test Drivers

Level N
Test Drivers Level N-1

Level N

Level N-1

Function Testing

1.

2.

Designed to exercise the code to its external specifications Testers not biased by knowledge of the programs design. Disadvantages: The need for explicitly stated requirements. Only cover a small portion of the possible test conditions.

Regression Testing

Test the effects of the newly introduced changes on all the previously integrated code. The common strategy is to accumulate a comprehensive regression bucket but also to define a subset. The full bucket is run only occasionally, but the subset is run against every spin. Disadvantages : To decide how much of a subset to use and which tests to select.

System Testing
Testing the fully integrated
applications including external peripherals in order to check how components interact with one another and with the system as a whole. This is also called End to End scenario testing. Verify thorough testing of every input in the application to check for desired outputs. Testing of the users experience with the application.

User Acceptance Testing

Analysis of Business Requirements Creation of UAT test plan Identify Test Scenarios Create UAT Test Cases Preparation of Test Data Run the Test cases Record the Results Confirm business objectives.

Implementation
A marketing company wishes to construct a
decision table to decide how to treat clients according to three characteristics: Gender, City Dweller, and age group: A(under 30), B(between 30 and 60), C(over 60). The company has four products (W, X, Y and Z) to test market. Product W will appeal to female city dwellers. Product X will appeal to old females. Product Y will appeal to Male middle aged shoppers who do not live in cities. Product Z will appeal to all but young city dwellers.

Solution : Decision Table based Testing

1. Identify Conditions & Values The three data attributes tested by the conditions in this problem are gender, with values M and F city dweller, with value Y and N age group, with values A, B, and C 2. Compute Maximum Number of Rules The maximum number of rules is 2 x 2 x 3 = 12 3. Identify Possible Actions The four actions are: market product W, X, Y, Z.

Solution (cntd)
4. Enter All Possible Rules The top of the table would look as follows: Note that all combinations of values are present.

Solution (cntd)
5. Define Actions for each Rule

Simplify the table

Look for impossible rules Having mutually exclusive conditions It would help if an action is associated with these Look for rules having same action pattern Combine rules where some of the conditions have same values and for the rest all values are covered Remember that if all the values of the indifferent attribute are not covered, the rules cannot be combined.

Developing Test Cases

Developing test cases using decision tables Conditions provide the inputs; can help in generating 1. Equivalence classes 2. Boundary values Actions provide the outputs. The rules thus formed lead to test cases.

Equivalence Partitioning
It is a method for deriving test cases. To use equivalence partitioning, you will need to perform the following steps: Identify the equivalence classes. 1. Program specifications should identify both valid and invalid inputs for the program. 2. Valid EC are chosen to represent valid inputs. 3. Invalid EC are chosen to represent invalid inputs. Design test cases.

Selecting Test Cases

Choose at least one test case from each class identified. For valid classes, choose test cases to cover as many equivalence classes as possible, until all valid classes have been covered. For invalid classes, choose test cases so that each covers one and only one invalid class, until all classes are covered.