STLC - Software Testing Life Cycle

There is a systematic cycle to software testing, although it varies from organization to organization Software Testing Life Cycle: Software testing life cycle or STLC refers to a comprehensive group of testing related actions specifying details of every action along with the specification of the best time to perform such actions. There can not be a standardized testing process across various organizations, however every organization involved in software development business, defines & follows some sort of testing life cycle. STLC by & large comprises of following Six Sequential Phases: 1) Planning of Tests 2) Analysis of Tests 3) Designing of Tests 4) Creation & Verification of Tests 5) Execution of Testing Cycles 6) Performance Testing, Documentation 7) Actions after Implementation Every company follows its own software testing life cycle to suit its own requirements, culture & available resources. The software testing life cycle cant be viewed in isolation, rather it interacts with the every phase of Software Development Life Cycle (SDLC). Prime focus of the software testing life cycle is on managing & controlling all activities of software testing. Testing might be manual testing or an automated testing using some tool. 1) Planning of Tests: In this phase a senior person like the project manager plans & identifies all the areas where testing efforts need to be applied, while operating within the boundaries of constraints like resources & budget. Unless judicious planning is done in the beginning, the result can be catastrophic with emergence of a poor quality product, dissatisfying the ultimate customer. Planning is not limited just to the initial phase, rather it is a continuous exercise extending till the end. During the planning stage, the team of senior level persons comes out with an outline of Testing Plan at High Level. The High Level Test Plan comprehensively describes the following: Scope of Testing : Defining the areas to be tested, identification of features to be covered during testing Identification of Approaches for Testing: Identification of approaches including types of testing

Defining Risks: Identification of different types of risks involved with the decided plan Identification of resources : Identification of resources like man, materials & machines which need to be deployed during Testing Time schedule: For performing the decided testing is aimed to deliver the end product as per the commitment made to the customer. Involvement of software testers begins in the planning phase of the software development life cycle. During the design phase, testers work with developers in determining what aspects of a design are testable and with what parameters those tests will work.

2) Analysis of Tests: Based upon the High Level Test Plan Document, further nitty-grittys covering the following are worked out. Identification of Types of Testing to be performed during various stages of Software Development Life Cycle. Identification of extent to which automation needs to be done. Identification of the time at which automation is to be carried out. Identification of documentation required for automated testing

The Software project cant be successful unless there is frequent interaction among various teams involved in Coding & Testing with the active involvement of the Project Managers, Business Analysts or even the customer. Any deficiencies in the decided test plans come to the surface, during such meetings of cross-functional teams. This provides an opportunity to have a rethinking & refining the strategies decided for testing. Based upon the customer requirements a detailed matrix for functional validation is prepared to cover the following areas: Ensure that each & every business requirement is getting covered through some test case or the other. Identification of the test cases best suited to the automated testing Identification of the areas to covered for performance testing and stress testing Carry out detailed review of documentation covering areas like Customer Requirements, Product Features & Specifications and Functional Design etc.

3) Designing of Tests: This phase involves the following: Further polishing of various Test Cases, Test Plans Revision & finalization of Matrix for Functional Validation. Finalization of risk assessment methodologies. In case line of automation is to be adopted, identification of test cases suitable for automation. Creation of scripts for Test cases decided for automation. Preparation of test data.

Establishing Unit testing Standards including defining acceptance criteria Revision & finalization of testing environment.

4) Construction and verification: This phase involves the following: Finalization of test plans and test cases Completion of script creation for test cased decided for automation. Completion of test plans for Performance testing & Stress testing. Providing technical support to the code developers in their effort directed towards unit testing. Bug logging in bug repository & preparation of detailed bug report. Performing Integration testing followed by reporting of defects detected if any.

5) Execution of Testing Cycles: This phase involves the following: Completion of test cycles by executing all the test cases till a predefined stage reaches or a stage of no detection of any more errors reach. This is an iterative process involving execution of Test Cases, Detection of Bugs, Bug Reporting, Modification of test cases if felt necessary, Fixing of bugs by the developers & finally repeating the testing cycles.

6) Performance Testing, Documentation & Actions after Implementation: This phase involves the following: Execution of test cases pertaining to performance testing & stress testing. Revision & finalization of test documentation Performing Acceptance testing, load testing followed by recovery testing Verification of the software application by simulating conditions of actual usage.

7) Actions after Implementation: This phase involves the following: Evaluation of the entire process of testing. Documentation of TGR (Things Gone Right) & TGW (Things Gone Wrong) reports. Identification of approaches to be followed in the event of occurrence of similar defects & problems in the future. Creation of comprehensive plans with a view to refine the process of Testing. Identification & fixing of newly cropped up errors on continuous basis.

Winding up of the test environment & restoration of all test equipment to the original base line conditions.

Life Cycle of Software Testing ( STLC ) Phase Planning of Tests Activities ($) Creation of a Test Plan of High Level ($) Creation of fully descriptive Test Plan ($) Creation of Matrix for Functional Validation ($) Creation of Test Cases Designing of Tests ($) Revision of Test Cases ($) Selection of Test Cases fit for automation Creation & Verification of Tests ($) Creation of scripts suitable for Test Cases for automation Refined Test Cases, Input Data Sets & Documents for Assessment of Risk Detailed Procedures for Testing, Testing Scripts, Test Reports & BugReports Detailed Test Reports & BugReports. Test Reports, Documentation on various metrics used during testing Outcome Refined Test Plans & Specifications Refined Test Plans, Test Cases & Matrix for Functional Validation

Analysis of Tests

SDLC 1) Initial Phase 2) Analysis Phase 3) Design phase 4) Coding Phase 5) Testing Phase 6) Delivery and Maintenance phase
Phase 1: Initial Phase: Involves Interaction with the customer and capturing all the requirements. Personnel Involved: 1) Business Annalist BA & 2) Engagement Manager - EM Process Involved: This phase begins with a technical meeting between the Business Analyst & the customer. The Business Analyst (BM) understands all the requirements from the customer & prepares comprehensive requirements document with the help of standard templates & check sheets used by the company. The requirements document forms the basis for further study & action by

Execution of Testing Cycles

($) Completion of Cycles of Testing

Performance Testing, Documentation

($) Execution of Test Cases related to performance tests & Stress Testing ($) Detailed documentation

Actions after Implementation

($) Evaluation of all Processes of Testing

Detailed Plans for improving the process of testing

the Engagement Manager (EM). The Engagement Manager tries to hunt for extra requirements if any, which might not be apparent immediately but might involve extra cost. To resolve ambiguous requirements if any, the Engagement Manager organizes demonstration of a prototype if necessary. Such action of quick demonstration of a prototype, which is nothing but a rapidly developed model for the customer is found extremely helpful to remove ambiguities from the requirements on one hand & help winning the confidence of the customer on the other. Ultimate objective of this phase is to prepare a most comprehensive customer requirements document. Many companies call it with different names like: # # # # # # BRS: Business Requirement Specification. FRS: Functional Requirement Specification. URS: User Requirement Specification. CRS: Client / Customer Requirement Specification. BDD: Business Design Document. BD : Business Document.

Depending upon the system followed by various companies, sometimes a single document is maintained, while some companies may go in for two documents. In this case, Broad / outlining information is recorded in the BRS (Business Requirement Specification), while detailed information is recorded in FRS (Functional Requirement Specification).

Phase 2: Analysis Phase: Involves Feasibility Study, Broad Planning, Technology selection, Requirement analysis etc. Personnel Involved: 1) System Annalist SA, 2) Project Manager PM & 3) Team Manager -TM Process Involved: To understand the nature of the program to be developed, the System Annalist focuses on understanding the information domain of the software, its necessary functions, behaviors, performance and necessary interfaces etc. The major objective of the analysis phase is to find out the need and to define the problem, which needs to be solved. The process covers four distinct analytical operations like: 1) Feasibility Study: Involves detailed study of all the requirements to ascertain as to whether it is possible to meet all the requirements or not. There may be a need for the development team to visit the customer place for study of their system. They development team may investigate the need for possible software automation within the boundaries of the given system. 2) Broad Planning: Involves an outline document on planning of all resource and time wise planning. 3) Technology Selection: Involves identification, analysis & compilation of various tools & technologies, which would be needed to accomplish the project successfully.

4) Requirement analysis: Involves identification & compilation of various requirements of human resources, hardware & software, which would be needed to accomplish the project successfully. Ultimate objective of this phase is to prepare a most comprehensive document called SRC or Software Requirement Specification.

Phase 3: Design Phase: Involves hard core design activities like 1) HLD - High Level Designing & 2) LLD - Low Level Designing Personnel Involved: 1) Chief Architect CA is responsible for High Level Designing, 2) Technical Lead TL is responsible for Low Level Designing Process Involved: The Chief Architect CA, starts The entire designing process. This involves subdivision of the entire project into various modules & the process is adequately explained with the help of flow charts & diagrams. Thereafter, the Technical Lead TL further subdivides these main Modules into sub modules with the help of Unified Modeling Language - UML. The technical lead is responsible for creating the Pseudo Code which is a comprehensive set of instructions written in English language & are meant for guiding the actual code developers. Ultimate objective of this phase is to prepare a Technical Design Document - TDD.

Phase 4: Coding Phase: Involves actual Programming or Coding. Personnel Involved: Code Developers or Programmers Process Involved: The design needs to be translated into a machine-readable form. There comes the process of code generation. With the help of Pseudo Code written by the Technical Lead TL during the Design Phase, Developers carry out the development of the actual source code. The entire coding process is bound by stipulated coding standards defined by the company. Ultimate objective of this phase is to prepare a Source Code Document - SCD.

Phase 5: Testing Phase: Involves rigorous testing of the application developed by the programmers. Personnel Involved: Testing Engineers. Process Involved: Different testing methodologies are practiced to detect the software bugs, which happen to take birth during the previous phases. Various testing tools and methodologies are available these days. Some companies create their own testing tools, which are custom made to suit their own development functions. # In fact the testing process begins right from the customer requirement document. A good test engineer starts his work by thorough study of the requirement

document. In case of any doubt arising during understanding of customer requirements, the test engineer raises a Review Report RR, which contains details of all the doubts. # Having clarified all the doubts & ambiguities, and after having a thorough understanding of the requirements, the test engineer shall write the test cases with the help of company specific test case templates. # Execution of the test cases upon the release of the first build by the developers. This follows recording of defects in a defect profile document. # The defect profile document is transferred to the developers for improvements / eradication of defects found by them. The testing process halts here till next build with improvements is released by the developers. # The entire testing process involving execution of test cases is repeated on the new build. This process of code improvement, subsequent testing on new builds & finding of defects continues again and again till the product is free from defects. # The testing process is decided to be stopped after gaining reasonable confidence of product being defect free Ultimate objective of this phase is to produce a Quality Product.

Phase 6: Delivery and Maintenance phase: Involves Installing the newly developed application under the client environment. Personnel Involved: Senior Testing Engineers who are entrusted the task of Deployment Engineers. Process Involved: # Delivery: After the code is appropriately tested, it is approved for release, it is deployed into a production environment. The deployment of application is done with the help of guidelines provided in the application deployment document. # Maintenance: After the deployment of the application, client may encounter many task-based problems. Maintaining and enhancing the software to cope up with newly discovered problems or new requirements can take far more time than the initial development of the software. It may be necessary to add some code, which does not fit the original design to correct an unforeseen problem. It may be that the customer is requesting more functionality, and some code can be added to take care of his requests.

The concept of Traceability Matrix is very important from the Testing perspective. It is document which maps requirements with test cases. By preparing Traceability matrix, we can ensure that we have covered all the required functionalities of the application in our test cases.

What is Traceability Matrix from Software Testing perspective?

The concept of Traceability Matrix is very important from the Testing perspective. It is document which maps requirements with test cases. By preparing Traceability matrix, we can ensure that we have covered all the required functionalities of the application in our test cases. Some of the features of the traceability matrix:

It is a method for tracing each requirement from its point of origin, through each development phase and work product, to the delivered product Can indicate through identifiers where the requirement is originated, specified, created, tested, and delivered Will indicate for each work product the requirement(s) this work product satisfies Facilitates communications, helping customer relationship management and commitment negotiation

Traceability matrix is the answer of the following basic questions of any Software Project:

How is it possible to ensure, for each phase of the lifecycle, that I have correctly accounted for all the customers needs? How can I ensure that the final software product will meet the customers needs? For example I have a functionality which checks if I put invalid password in the password field the application throws an error message Invalid password. Now we can only make sure this requirement is captured in the test case by traceability matrix.

Some more challenges we can overcome by Traceability matrix:

Demonstrate to the customer that the requested contents have been developed Ensure that all requirements are correct and included in the test plan and the test cases Ensure that developers are not creating features that no one has requested The system that is built may not have the necessary functionality to meet the customers and users needs and expectations. How to identify the missing parts? If there are modifications in the design specifications, there is no means of tracking the changes If there is no mapping of test cases to the requirements, it may result in missing a major defect in the system

The completed system may have Extra functionality that may have not been specified in the design specification, resulting in wastage of manpower, time and effort. If the code component that constitutes the customers high priority requirements is not known, then the areas that need to be worked first may not be known thereby decreasing the chances of shipping a useful product on schedule A seemingly simple request might involve changes to several parts of the system and if proper Traceability process is not followed, the evaluation of the work that may be needed to satisfy the request may not be correctly evaluated

Step by step process of creating a Traceability Matrix from requirements: step1: Identify all the testable requirements in granular level from various requirement specification documents. These documents vary from project to project. Typical requirements you need to capture are as follows: Used cases (all the flows are captured) Error Messages Business rules Functional rules SRS FRS So on example requirements: login functionality, generate report, update something etc. step2: In every project you must be creating test-cases to test the functionality as defined by the requirements. In this case you want to extend the traceability to those test-cases. In the example table below the test-cases are identified with a TC_ prefix. Put all those requirements in the top row of a spreadsheet. And use the right hand column of the spreadsheet to jot down all the test cases you have written for that particular requirement. In most of the cases you will have multiple test cases you have written to test one requirement. See the sample spreadsheet below:

Sample traceability matrix

Requireme Reqs nt Tested Identifiers

REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 REQ1 RE UC UC UC UC UC UC UC UC UC UC UC TECH TECH TEC 1.1 1.2 1.3 2.1 2.2 2.3.1 2.3.2 2.3.3 2.4 3.1 3.2 1.1 1.2 1.
3 2 3 1 1 1 1 1 1 2 3 1 1

Test Cases 321 Tested Implicitly 77 1 2

TC1.1.1 TC1.1.2

x x x

TC1.1.3 TC1.1.4 TC1.1.5 TC1.1.6 TC1.1.7 TC1.2.1 TC1.2.2 TC1.2.3 TC1.3.1 TC1.3.2 TC1.3.3 TC1.3.4 TC1.3.5 etc TC5.6.2

2 1 2 1 1 2 2 2 1 1 1 1 1

x x x x x x x x x x x x x x x x

step3: Put cross against each of the test case to each requirement if that particular test case is checking that particular requirement partially or completely. In the above table you can see REQ1 UC1.1 is checked by three test cases. (TC1.1.1, TC1.1.3, TC1.1.5). Another example of traceability matrix where requirement documents (use case) are mapped back to the test cases.

Change management through traceability matrix: It will be lot easier for you to track changes if you have a good traceability matrix in place. For example REQ1 UC1.1 we know upfront from the traceability matrix that what test cases I need to modify to incorporate those changes. In the above case we need to modify TC1.1.1, TC1.1.3 and TC1.1.5 only.