Software Testing Principles and Concepts

1. Vocabulary

Quality is segmented in to two Categories

Preventive Method
Detective Method

Producer Viewpoint: Quality of the products meets his requirements

Customer Viewpoint: Quality of the product is fit for use or meets the customer needs

1.1 Difference between QA and QC

Quality Assurance Quality Control

QA helps to establish the process
QA identify the weakness of the current process and
improve the process
QA is concerned with all the products where the
process is followed.
QA is also called as QC over QC and evaluates
whether QC is working
If there is no process there is no QA
QA is the set of systematic activities provide the
confident that the products met his specification and
satisfy the user needs
Quality policy is implemented through the
development and continues improvements of the
software development process
QC Relate with specific Product or
Service
QC verifies whether the specific feature
are available or not
QC identify the defect and correcting the
defects
QC is the responsibility to the
team/worker
QC is concerned with the specific product
QC is the process where product quality
is compared with the standard and ensure
that the product is developed based on
the standards
QC is start with the requirements and
ends when testing is completed.

Internal Auditing is an independent appraisal activity within organizations for the review
of operations and is the service to management.

1.2 Cost of Quality

The cost of quality is basically the amount spend to develop the product.

The cost of quality includes the additional costs spend to ensure that the products is
delivered with quality.

Prevention Costs

Money spends to prevent error before the product is build.

Money spend to ensure that to do the job right at the first time.
 It includes money spends on creating the testing process and standard, Learning
Tools, training provided to workers etc.

Appraisal Costs

Money spend to review the products against requirements.

It has been done after the product is build and before the products is delivered
to the customers.

It include money spend on inspection, testing and reviews

Failure Costs

Money spends once the products are implemented in the production.

It includes repairing of products and makes them to meet requirements.

1.3 Software Quality Factors

It mainly deals with the Risk Factors

What will happened if the features which is needed not available in the
applications.

Not Easy to Use

There are 11 Software quality factors

Reliability  Perform intended function with required precision

Maintainability - Effort of testing the application any changes are made
Testability - Effort for testing the applications
Flexibity  Effort required modifying the program.
Portability  Effort to transfer from one configuration to another
Integrity  Unauthorized person cannot be accessed.
Reusability  Reuse some set of functions
Interoperability  Couple one system with other
Correctness - Fulfill the user’s Mission Objectives
Efficiency  resources to perform a function.
Usability  Effort for learning

It include the following survey

Consider Basic Characteristics of the Application

Consider Life Cycle Implications
Perform Trade off among the tentative list of software quality factors

Identify the most important software quality factors

Provide explanations for choices.
 1.4 How Quality is defined.

Quality is based upon customer satisfaction; each organization is needed to define

the quality before it was achieved.

There are five prospective for quality

Product Based  Product has all the features

Value Based  Acceptable Cost
User Based  Fit for Use
Transcendent  I know it when I see it
Development and manufacturing base  Meet his requirements.

Quality

Quality is all about treating the client with respect by meeting the customer expectations
by satisfying the customer needs by delivering the right product.

Quality in Fact Quality in Perception

Doing the Right Thing Delivering the Right Product
Doing in Right Way Satisfy the Customer Needs
Doing it for First time Meeting the Customer Expectation
Doing it on Time Treating Every customer with Respect

Excellence

It is the measure or degree of quality. It is very important and it also the starting point for
any management to implement the quality policy.

The common trend of today quality is mainly focus on customer and more importantly
customer satisfaction.

There are two types of Customers

External Customers:

The customer you are using your products provided by the organization.
It basically identifies the customer and satisfies the requirements.

Internal Customers:

A Person or groups which provide the results of any work. It is either related with
products, or individual performance etc.

It includes subordinates team members and other people within the organization.

All organization will focus on both Internal and External Customers.

 1.5 Why do we test software?

Developers are not Good Testers

The Limitation that the person who developed itself has testing based on the requirements
as follows.

1. Misunderstanding cannot be deductible

2. Improper use of Software development Process.
3. Due to the change in the requirements the system specification is keep on changing in
spite of that they may be certain functionality may missed which was not noticed.
4. Developers blindly thinking what they developed are correct.
5. Improve the developmental process.

1.7 Quality Software

As far as developers are concerned they are focusing whether the product is built based on
the specification.

As far as Customer is concerned they are focusing whether the products is fit for use and
met the user needs.

There will be gap in between us. As software testers we need to reduce the gap in between
and make the products as consistent one.

The constants product can achieve by interacting with the clients on regular intervals.
They also need to include in the meeting and ensure that the products is developed based
on user needs.

1.8 Why Development process produce Defects

Developers won’t take Variability in to account why developing the product.

Variability is the enemy of quality.

As a testers need to be reduce the Variability and improve the quality of the product.

The concept of Measuring and reducing Variability is called Statistical Process Control (SPC)

The application is stable if the process remains constant over time.

The Limits is needed to be within the range all the times.

The Important of measuring the variability is used to identify the weakness of the current
process and also helps to improve the process.

It also helps the process to be consistent in all the time.

Special Causes of Variability is not included in the process it occurs because of unique
circumstances.

Common Cause of Variation is nothing but the collection of small variation which affect the
major concerns.
 With he help of Statistical results we can identify he current process and take immediate
step to improve the process.

Some of the examples for Variability

Machine

Office Equipment, Software , Hardware

People component

Training, experience Attitude

Method Component

Procedure, Policies

Environment

Temperature, Lighting, Humidity

Process be in control

The variation is calculated based on standard deviation and mean. If the standard
deviation and mean is constant over time the process is stable.

Brain Joiner summarized the special causes of variation as follows.

Process input and condition is contribute the variability of process outputs.

Each special causes can causes small or large in the total variation of the process
outputs.
The varability is identified by the use of control charts

Elimate the special causes of variation

Take up the immediate steps when variability occurred

Do not make fundamental changes in the process
User early warning throughout your operation.

Why Testers need to know SPC

Testing is nothing but the Measurement process. Based on the statistical output we can able to
analysis the quality of the product which helps to identify the weakness and improve the quality of
the product.

1.9 Reducing the frequency of defects in the software development

The capability maturity model is the algorithm which helps to identify the more effective and
efficient product.
They identify five maturity model level 1 to Level 5

Level 5 --- Has less Variability

Level 1  has more Variability

The cost spend in level 1 is 10 times more than Level 5.

If the cost spend is 1000 in level 1 company may 10 in Level 5 Organization

Level 1 (Adhoc)

--- Unstructured
--- No proper Discipline and Culture
--- Process was not defined.
--- Process was differing from project to project.
--- Workers basically concentrate on there personal agenda rather than organization
Mission.

Level 2 - Control

--- Structured
--- Proper Discipline and Culture
--- Process is well defined.
--- Verification and Validation to check the product standard
--- Concentrate in building and improve the work process.

Level 3 – Core Competency

--- Competency is defined and then the organization is build

--- Each competency has its own responsibility to improve them.
--- Training need to provide to each individual so that they are capable to handle.
--- Well Structured
--- Process is well defined.
--- Proper discipline and culture.
--concentrate in building and improve the work process.

Level 4 Predictable

--- Measurement is the key factor and evaluates the individual performance.
--- Management is the coaches to all the individual by identify the obstacles facing day
to day activities and take necessary steps.
--- Competency is defined and then the organization is build
--- Each competency has its own responsibility to improve them.
--- Training need to provide to each individual so that they are capable to handle.
--- Well Structured
--- Process is well defined.
--- Proper discipline and culture.
--concentrate in building and improve the work process.

Level 5 Innovative

--- True leader in the Industry

--- Measure themselves through Benchmarking
--- Constantly focusing on improving.
 --- New Technology is Introduced
--- Continuous learning occurs in both individual as well as organization.

Multiple Roles of Software Testers

1.10 People Relationship

- Testing is finally recognized as a profession which needs specialized skills set and
qualifications.

- Also it is convinced testing will not start once the development is completed.

- Testing needs to be start in every stage of the software development Life Cycle.

- Testers need to validate whether the requirements specification documents is

developed based on the needs of the organization.

- Testers need to be ensuring that the design documents developed based on the
requirements specifications.

- Testers need to be ensure that the test cases and test plan was created based on the
requirements specifications.

- Testers need to be ensuring that the defects need to find out before the test cycle starts
so that the cost spends will gradually decreases.

- Finding out defects in the later stage of the product will leads the cost spend to recify
the defects is very costly.

The top 10 people challenges have been identified

Training in Testing
Good Relationship with Developers.
Good Relationship with Customers.
Managers need to understand about testing.
Using of Tools.
time on Testing
Fighting a lose-lose situation
Having to say No
Testing “Over the wall” Software
Having to say “no”

1.11 Scope of Testing

 As far as tester is concerned, needs to be test in such that it is fit for use and meet he
user needs.

 Tester’s needs to test the product based on customer prospective.

 Testers need to take up the responsibility to make the bug free products.

 It is testers’ responsibility to ensure to certify that the products are ready for
production.
  Testers need to find out the defects in the early stage, defects found out in the later
stage of the build may cause difficult to rectify the defects and also cost faster is very
high.

For example: in case there is the issue in the design stage and it was not identified and
the coding was developed and the testing has done and implemented in the production. In the
later stage once they identified in the production. Erroneous cost need to spend to resolve the
issue. Designed need to be change based on organization needs and coding need to be altered,
test case need to be change , risks also need to be considered so that the new features doesn’t
affects the other features before it is implemented in the production.

Testers need to involves in all the phases

Requirements  Design Coding  Testing  Implementation  Maintenance.

Testing Constraints

Budget and Schedule Constraints

Budget and Schedule can limit the testers to complete their test plan.

The cost is dynamically increases as defects found out in the later stage in the test
phase.

It is Testers Responsibility to ensure to find out the defects in the early stage of the life
cycle so that help to reduce the number spend to resolve the problem.

As the cost of testing increases the number of uncovered defects decreases.

The cost of uncovered defects is increases as the cost spend of the testing is less.

Most problem associate with testing occur in one of the following causes

Fail to define test objects.

Test the application in the wrong phase
Ineffective test objectives

Software Risks

Incorrect results will be produced

Unauthorized transactions will be accepted by the system
Application is not easy to Use
Application can’t able to maintain
It is difficult to operate
Performance level is not acceptable.

In any of the Risk occurs it affect the substantial loss in the organization.

The decision is need to take how much risk is acceptable and it is clearly mentioned in
the test plan.
Defects are identified in the software because of following Circumstances

IT Improper Requirements.
User Provide the Wrong Requirements
Requirements are wrongly Recorded
Design Specifications is incorrect.
Program Specifications is Incorrect.
Entire Coverage is not tested properly
Time pressures
Inadequate Testers
Inadequate Developers

Testers need to identify two types of defects

Variance from specifications  Defects from perspective of the product

Variance from what is desired  Defects from customer prospective.

Life Cycle Testing

Life cycle testing involves continuous testing of the application even after software plan
are completed and the tested system is implemented.

Test Matrices

The test matrix shows the interrelationship between functional events and tests.

The completed test matrix will contain the condition that tested during the test process
and the corresponding action.

The left side of the matrix shows the functional events and the top identifies the test occur
on those events.

Cascading Test Matrices

The test that occur on functional events vary based on event themselves.

Instead of maintaining one single functional events and tests we can decomposed in to
two or more functional events. Based on the output of the one functional events we can process
the other functional events.

It has the following 2 advantages

Tests can be customized in to functional events

Based on the output of the one functional events will create a other functional
events which shows the relationship between the events.
Independent Testing

The primary responsibility of the independent testing is to ensure that quality is measure
accurately.

As far as independent testing exists there are mainly responsible for system testing and
oversight of Acceptance Testing .

As a testers the individual Must

Understand the System.

Understand the Technology it was developed

They are also involved in other types of testing such as load testing and stress testing etc.

Test manager and Testers are involved in Independent testing.

Test Manager

Test planning and estimation.

Designing the Test strategy
Reviewing analysis and design artifacts
Managing the test effort
Overseeing the Acceptance Criteria.
Controlling the Test Members.

Testers

Developing the test cases

Execute the test cases
Defect tracking and reporting.
Review Analysis and design artifacts.
Involved in System testing and Regression Testing

1.18 Tester’s Workbench

Process is the set of activities to accomplish the task.

Both development and testing are maintaining the process.
The outcome of the process is a product or a service.

PDCA is the conceptual view of process

Workbench is a more practical illustration of the process

Plan Do Check Act (PDCA)

Plan  Derive a plan

Do  Execute the plan
Check  Check and ensure that we are working as per the plan.
Act  Make the necessary action or alternative so that at any point of time we are meting
the objective.
Work Bench Process

The working is nothing but the tasks , phases or stages.

Each task has three components Input  Process  Deliverable.

Before the task starts we need to ensure that all the entry criteria is available as per the test

Procedure

Procedure to do work  basically check what are the task need to be accomplished

Procedure to check work - Basically ensure that whether we met the specifications and
standards.

Rework  If there is any issues it moved to rework. If it is not satisfied the standards.

Release  If there is no problem it is moved to deliverables.

1.19 The V Concepts of Testing

It is the continuous testing of the system during the developmental process.

Both Developmental activities and testing activities will carried out simultaneously

Development team and testing team will maintain the separate methalogies and met the
objective.

At certain point of time in the development process the testing process will come in to
picture and start finding defects in the system.

During the acceptance criteria both development and testing team will work closely and
properly integrated in to production environment.

Development Testers

Understand the requirements Understand the development Plan

Bulid the software Develop the test plan

Install the software Understand the requirements
Maintain the software Understand the design
Understand the low level design
Execute and record results
Report the test results
Do Acceptance testing
Install the application in the production
Enhancement
1.20 Structural and Functional Technique

1.21 Verification and Validation.

1.22 Static versus Dynamic Testing

Structural and Functional Technique

Both Structural and Functional Technique is used to ensure adequate testing

Structural analysis basically test the uncover error occur during the coding of the program.

Functional analysis basically test he uncover occur during implementing requirements

and design specifications.

Functional testing basically concern about the results but not the processing.

Structural testing is basically concern both the results and also the process.

Structural testing is used in all the phases where design , requirements and algorithm is
discussed.

The main objective of the Structural testing to ensure that the functionality is working fine
and the product is technically good enough to implement in the real environment.

Functional testing is some times called as black box testing, no need to know about the
coding of the program.

Structural testing is some times called as white box testing because knowledge of code is
very much essential. We need the understand the code written by other users.

Various Structural Testing are

Stress Testing
Execution Testing
Operations Testing
Recovery Testing
Compliance Testing
Security Testing

Stress Testing

• Basically test the application to ensure how system is behave when large volume
of data interact with the system.

• The objective of the stress testing is basically stress the application and check
the applications before it is put in to the production environment.

• Normal and abnormal of data was processed during the specific time frame

• Test data can create either with the data used in the production environment or
the set of data can create on your own.
 • We can also check whether the sufficient disk space is allocated to the
application.

• We can also check whether any communication link failure come across at the
time of the processing the transactions.

• People you are involving of entering instructions need to the end user you is
going to use the application one it is put in to production.

• Stress testing need to break the system by overloading the large number of
transactions.

• The disadvantage of stress testing is the amount to time to use to prepare for the
test and the amount of resources used for it. However it will helps to take up the
decision to whether we make the application live based on the results which will
help to reduce the risk.

• It will be done before UAT towards end of the developmental phase.

Execution Testing

 Helps to determine whether the system achieve the desired level of

proficiency in the production status.

 Helps to verify the turnaround times and response times as well as

design performance.

 The objective is used to check the performance of the system.

 With the help of execution testing. We can

• Determine the performance of the system.

• Optimum usage of software and hardware
• Determine the Response time.
• Determine the wait around time.
•
` Execution Test Examples

• Check the hardware and software utilization.

• Check the behavior of each component in the application and
determine the program code is effectively used.
• Calculate the response time and turn around time of the
application.

Execution Test can be used in early stage of the development process and
check the performance of the code is effectively used.

Any phase of Development life cycle.

 Recovery Testing

 Ability to restart the operations after the integrity of the application is lost.

 Objective of the recovery testing is to ensure that the operations can

continue
 Even though the disaster is happening.

 Recovery testing includes

• Storage of data in the preserved location.

• Documents need to maintain properly about recovery.
• Assign the professional testers involving in Recovery process.
• Develop a recovery tool and make it available all the times.

 Recovery testing is normally takes care by professional testers, System

analysts you know entire flow of the applications.

 It is advice that the operational staff need not involved in recovery testing
and only professionals need to be involved.

 Recovery can be conducted in two modes.

• Either with the recovery tools and the set of the methods and
procedures to resolve it.

• Once the system is developed and the failure is introduced in to

the system and the ability to recover it.

The user should estimate the potential loss inability to recover over the time
spans. Ability not to recovered may lead to the business loss.

The potential loss is calculated in such that the amount of resource used for
recovery testing.

Operations Testing Technique

o After testing and the application is put in to the productions and the operational `
staff is going to use the application.

o Objective

 Prepare the detail documents which contain the details overall of the
application.
 Provide the complete training to the Users.

o Operating testing involved two stages.

  Process
 Execution of the process.

 Process: With the help of requirements documents, design documents

we can able to understand the overall architecture of the product.

 Execution of the process is basically determines how it has been

implemented so that the end user understand the system and fit for use
and met the user needs.

o Operational testing is basically done before the production start and it is each
end user responsibility to understand the system properly before the productions
start.

Compliance Testing Technique

Verifies whether applications are developed based on the IT `Standards, `

procedure and guidelines.

It is very importance Compliance testing need to be done in all the phases of the `
developmental process.

Objectives

Determine whether the system development and maintenance

methodologies is followed.

Ensure Guidelines and standards

Compliance testing can achieve in which documents of the standards need

to be compared with the applications and ensure that at any point of time
and the application is smart enough as it follows all the guidelines properly.

Once the program is developed the group of developers can sit together
and review the codes and ensure that the programming code is developed
based on the IT Standard and guidelines.

Security Testing Techniques

Security testing is basically secure the confidential information’s so that the

information cannot able to access by third party.

Managements need to be very much clearly which the confidential information is.
Because the improper disclosure will make the business loss.

The objectives of the security defects is very much difficult to identify

Objective

Identify the security information and also

 Functional System Testing Technique

Basically check the system requirements and specifications are achieved

There are various Techniques as far as Functional Testing is concerned.

Requirements Testing

Basically verify the system is working as far the specifications over the continuous period of
time.

The main objectives of the requirements testing is to

Implements user requirements.

Fit for use based on User Needs.

Maintain corrections of the programs over the period of time and ensure it is capable to
make it live.

Ensure that the applications are developed based on Organization Policy and Guidelines.

The process is start with the requirements and continues in each and every phase of the
system developments life cycle.

Regression Testing

Basically retest the application and ensure that the applications are working as per the
requirements even though there is the change made in another part of the applications.

Once the application had been implemented in the production and the new features had
requested by the client and the developers had develop it, Regression testing is done on the
applications before it is put in to the applications to ensure that the new features apply on the
applications couldn’t affect the existing feature already have in the application.

At the time of creating the new products in each stage of developing the new product new
features are getting added and however with the help of regression testing we can able to check
that the new features doesn’t affect the existing features in the application.

Each and every time existing test cases which is developed earlier is tested against the
applications.

Error Handling Testing Techniques.

 The main objective of the Error handling testing techniques is to ensure that the
applications are capable to handling incorrect Transactions.
  The application is smart enough to identify the incorrect transactions and approximate
decision need to take and intimate the end user.

 Normal testing is basically test the application is working properly as per the
requirements but error handling testing is exactly the opposite manner.

 Error handling testing is basically done by Experience persons who had the full
knowledge of the domain.

 Error handling testing can generate in such that the improper transactions is done in-
between the proper transactions and check how the system is behaved during improper
transactions and identify the problems.

 Use the improper master data and check the application whether it is capable to identify
the problem and informed to the Users.

Integration Testing

Integration testing is basically check the applications when two or more modules
combines together and ensure that it is working based on the requirements.

Initially once the application is developed it is develop individually and test the application
whether it is satisfy the requirements.

As the applications which is implemented in the production is nothing but the collection of
modules as the testers we need to ensure that the application is working as per the user needs
once all the modules are integrated with the single application.

Control Testing

Control are designed to reduce risks; risks need to be identified.

Create a scenario and ensure that the risk is created and check whether control can easily
reduced the risks.

Control oriented people can frequently do control testing. Like error handling it required
negative look on the application to ensure that what can go wrong.

Parallel Testing Techniques

The main objective of parallel testing is to ensure that the new application is consistent
with the processing of previous application or version of the application.

Old system and new system are run and the results compared to detect unplanned
differences

It is easy to use
It is easy to maintain
It is easy to operate when compared to old version of the software.

As long as the staff are very much comfortable in the new version, they will continue with
the old version.
 The change of new version can happened either in a day or a week or a month or
sometimes it takes a year.

It all based on the people you are involving and the performance of the application.

No Verification Validation
.
1. Verification is a STATIC testing procedure. Validation is DYNAMIC testing
procedure.
2. It involves verifying the requirements, Validation involves actual testing of
detailed design documents, test plans, the product as per the test
walkthroughs and inspections of various plan (unit test, integration test,
documents produced system test and acceptance test etc).
during the development and testing process.
3. It is a PREVENTIVE procedure. It is a CORRECTIVE procedure.
4. Are we building the product RIGHT? Are we building the RIGHT product?

5. It involves more then two to three persons
and is a group activity.
6. It is also called Human testing, since it
involves finding the errors by persons
participating in a review or walk through.
7. Verification occurs on Requirements, Design Validation occurs only on code and
and code.
8. Verification is made both in the Executable Validation is done only on Executable
and Non Executable forms of a work product forms of a work product.
9. Verification finds errors early in the
requirement & design phase and hence
reduces the cost of errors.
10. An effective tool for verification tool is a
Checklist.
11. It requires cooperation and scheduling of
meetings and discussions.
12. Verification tasks include:
1) Planning
2) Execution
13. Verification activities include:
1) Requirements Verification
2) Functional design verification
3) Internal Design Verification
4) Code Verification
14. Verification deliverables (work products)
are:
1) Verification test plan
It involves the testers and sometimes
user.
It is also called Computer testing,
since errors are found out by testing
the software on a computer.
the executable application.
Validation finds errors only during the
testing stage and hence cost of errors
reduced is less than Verification.
Various manual and automated test
tools are available for Validation.
It is to check that the product
satisfies the requirements and is
accepted by the user.
Validation tasks include:
1) Planning
2) Testware Development
3) Test Execution
4) Testware Maintenance
Validation activities include:
1) Unit testing
2) Usability testing
3) Function testing
4) System testing
5) Acceptance testing
Validation deliverables are:
1) Test plan
2) Test Design Specification
 2) Inspection report 3) Test Case Specification
3) Verification test report 4) Test Procedure Specification
5) Test log
6) Test incident report

Static versus Dynamic Testing

Static testing is performed using the software documentation.

The Code is not executing during static testing.

Verification

Feasibility Review
Requirements Review

Validation

System Testing
Unit Testing
Integration Testing
User Acceptance Testing

White Box Testing

Statement Coverage
Decision Coverage
Condition Coverage
Decision / Condition Coverage
Multiple Condition Coverage

Black Box Testing

Equivalence Partitioning

Subset of Data represent in the larger class

Technique for testing equivalence class rather than undertaking exhaustive
testing of each value of the larger class.

If the text box accept only between 10 to 100 check condition will be

< 10
> 100
Between 10 and 100

Boundary value analysis

Checking on Boundary
Checking Lower Boundary
Checking Upper Boundary
 Error guessing

Test can be done based on the experience of the tester

If there is the date fields we can test by input Feb. 30 and check the application.

Incremental Testing

Top  Stub
Down  Drivers
Desk Checking and Peer Review

Desk Checking is the most traditional means for analyzing the problem.

In order to Improve the efficiency of desk checking it is important that the program review
the code against design specifications.

It is not easy to find out error on our own code , it is good to review the other
programming code. And the same other person can review your code.

Peer review will tends to increase the quality of the software production.

A Panel is set up for review the sample code on a regular basis and provide feedback to
the individual programmer.