Software Testing

Lecture 1
Introduction

Dr. Fan Hongfei

27 Feb 2019
Discussion
• What is your understanding about software
testing?
• Do you apply software testing during software
development process?
• How much effort do you spend on testing
during the development process?
• What strategies and approaches do you use
for testing software?

2
Planned Topics
1. Fundamentals of testing
 Terms, test process, general principles
2. Testing in the software life cycle
 Software development models, test levels, test types
3. Static test
4. Dynamic analysis - test design techniques
 Black box, white box
5. Usability testing
6. Test management
 Organization, planning, management
7. Test tools
 JUnit, Selenium
3
Textbook
• SOFTWARE TESTING FOUNDATIONS, 4TH
EDITION

4
Motivation
• Industry product: parts and final product
examined to make sure they fulfill the given
requirements
• For software, more difficult
– Immaterial, not a physical product
– Direct examination is impossible
– Only way: reading (reviewing), testing
• Software testing is important and difficult

5
Failure, Fault, Defect, Bug and Error
• Failure/problem/issue/incident
– A given requirement is not fulfilled, or an
expectation is not adequately met
• Functional, non-functional

• Defect/fault/bug
– Being present from the time the software was
developed
• Defect masking

• Error/mistake
– Made by a person (why?)
6
Threats of Software Failure
• For companies
– Damaged reputation for quality
– High or unpredictable maintenance costs
– Unexpected delays in release cycles
– Lack of confidence in system
– Lawsuits
• For environment
– Pollution, waste
• For people, societies and states
– Lost jobs/lives/rights/missions/wars
7
Testing vs. Debugging
• For correcting a defect, it must be localized
– Correcting defects may introduce new defects
• We need to repeat test cases, and conduct even more cases

• Testing: detection of failures

• Debugging: localizing and correcting faults

• Testing: the whole process of systematically

executing programs to demonstrate the correct
implementation of the requirements, to increase
confidence, and to detect failures 8
Testing Terms
• Test object
• Test data
• Test planning
• Test design
• Test implementation
• Test analysis
• Test run/test suite
• Test case
– Test conditions (preconditions, inputs, expected
outputs/behavior)
• Test scenario
9
Naming Tests
• Test objective/test type
– Functional test
• Test technique
– Specification-based test
• Test object
– GUI test, database test
• Test level
– System test
• Test person
– Developer test, user acceptance test
• Text extent
– Partial regression test, full test
10
Software Quality
• Software testing contributes to improvement of
software quality
• Software quality: more than just eliminating failures
– Functionality: adequacy, accuracy, interoperability,
correctness, security
– Reliability: maturity, fault tolerance, recoverability
– Usability: understandability, ease of learning, operability,
attractiveness, compliance to xyz
– Efficiency: required time, consumption of resources
– Maintainability: analyzability, changeability, stability,
testability
– Portability: adaptability, ease of installation, conformity,
interchangeability
• Quality characteristics must be prioritized
11
Some Facts
• No large software system is bug free
• Testing cannot produce absence of defects
– Even if all the executed test cases pass

12
Test Effort
• Complete/exhaustive testing is impossible
• Only possible to consider a part of all
imaginable cases
– Even so, testing still accounts for a large portion
of software development effort (25%~50%)
• Is testing effort affordable and justifiable?
– Depending on the estimation of risk (probability
of occurrence and expected amount of damage)
– Testing should continue as long as the costs of
finding and correcting a defect are lower than the
costs of failure
13
Test Effort (cont.)
• Defining test intensity and test extent
depending on risk
• Tests must be designed and executed in a
structured and systematic way
• For test managers
– A well-planned and efficient strategy
– A fundamental test process

14
Exercise
• The triangle test case design
– A program accepts three integers representing
the lengths of a triangle’s sides
– It outputs “scalene” (no equal sides), “isosceles”
(two equal sides), or “equilateral” (three equal
sides)
– Try to design test cases

15