Domain 6

Verification
and
Validation
Verification Concepts

“Verification concepts is the

understanding of principles,
rationale, rules, participant roles

and the psychologies of various

techniques used to evaluate
1
systems during development.”
CBOK Domain 6
Validation Concepts

“Validation typically involves

actual testing and takes place
2
after verifications are completed.”
www.Softwareqatest.com
Verification Techniques

Audits
• An independent assessment of a
project
• To verify whether or not the project
is in compliance with appropriate
policies, procedures, standards,
contractual specifications
• An audit may include operational
aspects of
the project
Verification Techniques

Reviews and Inspections

•To determine whether or not to
continue
development
• To identify defects in the product early
in the
life cycle.
Reviews and Inspections
Types of Reviews
1. In-Process Reviews
2. Milestone Reviews (also called)
Decision-point/Phase-end Reviews

4. Test Readiness Review

5. Test Completion Review

3. Post Implementation Reviews (also

called)
Post Mortem
NOTE: CBOK has Test Readiness Review and Test Completion
Review inside Decision-point/Phase-end Reviews and as separate
types of reviews. – Replaced Milestone Reviews with Decision-
point/Phase-end reviews
Types of Reviews
In-Process
• Assess progress towards
requirements
• During a specific period of the
development cycle – like design
period
• Limited to a segment of the product
• Used to find defects in the work
product
and the work process
• Catches defects early – where they
Types of Reviews
Decision-point & Phase-end
• Review of products and processes
near
the completion of each phase of
development
• Decisions for proceeding with
development are based on cost,
schedule,
risk, progress, readiness for next
phase
• Also referred to as Milestone Review
• Contains Requirements, Critical
Types of Reviews
Decision-point & Phase-end

Software Requirements Review

• Requirements documented
• Baseline established
• Analysis areas identified
• Software development plan
• Test plan
• Configuration management plan
derived
Types of Reviews
Decision-point & Phase-end

Critical Design Review

• Baselines the detailed design
specification
• Test cases are reviewed and
approved
• Usually, coding will begin at the
close of
this phase.
Types of Reviews
Decision-point & Phase-end

Test Readiness Reviews

• Performed when the appropriate
modules are near completion
• Determines whether or not testing
should progress based on a review
of entrance and exit criteria
• Determines the readiness of the
application/project for system and
acceptance testing
Types of Reviews
Decision-point & Phase-end

Test Completion Reviews

• Determine the state of the
software
product
Types of Reviews
Decision-point & Phase-end

Important Notes
• Completion of phase-end review
signals
beginning of next phase
• In iterative development
methodologies
each analysis/design “package” or
segment of the application may be
in
different phases simultaneously
• Must ensure iterations are
Types of Reviews
Post Implementation Reviews

• Also known as “Postmortems”

• Review/evaluation of the product
that
includes planned vs. actual
development
results and compliance with
requirements
• Used for process improvement of
software
development
•
Reviews and Inspections
Classes of Reviews
1. Informal Review

2. Semiformal Review

3. Formal Review
Classes of Reviews
Informal
• Also called peer-reviews
• Generally one-on-one meeting
between
author of a work product and a peer
• Initiated as a request for input
• No agenda
• Results are not formally reported
• Occur as needed through out each
phase
Classes of Reviews
Semiformal
• Facilitated by the author
• Presentation is made with comment
at the
end
• Presentation is made with comment
made
throughout
• Issues raised are captured and
published
in a report distributed to participants
Classes of Reviews
Formal
• Facilitated by a moderator (not
author)
• Moderator is assisted by a recorder
• Defects are recorded and assigned
• Meeting is planned
• Materials are distributed beforehand
• Participants are prepared- their
preparedness dictates the
effectiveness of
the review
Classes of Reviews
Formal - continued

• Full participation by all members of

the
reviewing team is required
• A formal report captures issues
raised and
is distributed to participants and
management
• Defects found are tracked through
the
defect tracking system and followed
through to resolution
Review Rules

1. The product is reviewed, not the

producer

2. Defects and issues are identified, not

corrected

3. All members of the reviewing team

are responsible for the results of the
review
Review Notes
“Stage Containment”: identifying defects in
the stage in which they were created, rather
than in later testing stages.
Reviews are generally greater than 65%
effective
Testing is often less than 30% effective
The earlier defects are found the less
expensive they are to correct
In addition to learning about a specific
product/project, team members are exposed
to a variety of approaches to technical issues
Provides training in and enforces the use of
Participant Roles
Management of V & V

1. Prepare the plans for execution of

the process
2. Initiate the implementation of the
plan
3. Monitor the execution of the plan
Participant Roles
Management of V & V

4. Analyze problems discovered

during the execution of the plan
5. Report progress of the processes
6. Ensure products satisfy
requirements
7. Assess evaluation results
8. Determine whether a task is
complete
9. Check the results for completeness
V & V Manager Role
Tasks and Responsibilities
• Create the Software Verification and
Validation plan
• Baseline Change Assessment
• Conduct Management Review of V
&V
• Management and Technical Review
Support
• Interface with Organizational and
Supporting Processes
V & V Manager Role
Software Dev. V & V Activities

1. Conceptualization
2. Requirements Analysis
3. Design
4. Coding
5. Integration
6. Testing
7. Installation
8. Acceptance
Software Dev. Activities
Software Dev. V & V Common
Tasks
• Criticality analysis
• Traceability analysis
• Hazard analysis
• Risk Analysis
Software Dev. Activities
Conceptualization
• Define a specific implementation
solution
to solve the user’s needs
• Architect selected
• System requirements defined and
packaged for analysis
• Objectives: to verify the allocation of
system requirements, validate the
selected
solution, and ensure that no false
assumptions were made
Software Dev. Activities
Conceptualization - Unique
Tasks
• Evaluate Concept Documentation
• Analyze Hardware/Software/User
Requirements Allocation
Software Dev. Activities
Analysis
• Refine and analyze the functional
and
performance requirements
• Interfaces external to the software
• Qualification requirements
• Safety and security requirements
• Human factors engineering
• Data definition
• User documentation for the software
Software Dev. Activities
Analysis - continued

• User operation and execution

requirements
• User maintenance requirements
• Object Oriented – define use cases
• Object Oriented – class diagram
created
• Object Oriented – sequence diagrams
• Objectives: Ensure the correctness,
completeness, accuracy, testability
and
Software Dev. Activities
Analysis – Unique Tasks
• Software Requirements Evaluation
• Interface Analysis
• Acceptance Test Procedure
Generation and
Verification
• System Test Plan Generation and
Verification
• Acceptance Test Plan Generation and
Verification
• Configuration Management
Software Dev. Activities
Design
• Address software architectural design
• Address software detailed design
• Includes databases
• Includes interfaces (external to the
software, between components and
between software units)
• Objectives: to demonstrate that the
design
is correct, accurate, complete
transformation
Software Dev. Activities
Design – Unique Tasks
• Software Design Evaluation
• Component Test Plan Generation and
Verification
• Integration Test Plan Generation and
Verification
• Test Design Generation and
Verification
Software Dev. Activities
Implementation
• Transform design into code
• Transform design into database
structures
• Transform design into related
machine
executable representations
• Addresses software coding
• Addresses unit testing
• Objectives: verify and validate that
the
Software Dev. Activities
Implementation – Unique Tasks
• Source Code and Source Code
Documentation Evaluation
• Interface Analysis
• Test Case Generation and
Verification
• Test Procedure Generation and
Verification
• Component (unit) Test Execution and

Verification
Software Dev. Activities
Testing
• Includes component integration
• Includes system testing
• Includes system integration
• Includes user acceptance testing
• Objectives: ensure that the functional
and
supplemental requirements are
satisfied by
execution of integration, system, and
acceptance tests.
Software Dev. Activities
Testing – Unique Tasks
• Integration Test Execution and
Verification
• System Test Execution and
Verification
• Acceptance Test and Verification
Software Dev. Activities
Installation & Checkout
• Includes the installation of the
software
product in the target environment
• Includes the acquirer’s (user’s)
acceptance
review and/or testing
• Objectives: to verify and validate the
correctness of the software
installation in
the target environment
Software Dev. Activities
Installation & Checkout –
Unique Tasks

• Installation Configuration Audit

• Installation Checkout
• V & V Final Report Generation
Software Dev. Activities
Operation
• Support the use of the software by the
end
user in an operational environment
• Addresses Operational testing,
• System operation
• And user support
• Objectives: to evaluate new
constraints on
the system, assess proposed changes
and
Software Dev. Activities
Operation – Unique Tasks
• Evaluate New Constraints
• Proposed Change Assessment
• Operation Procedures Evaluation
Software Dev. Activities
Maintenance
• Covers modifications (corrective,
adaptive,
perceptive)
• Address migration
-The movement of software to a new
operating system
• Address retirement of software
-The withdrawal of support by the
operation and maintenance
organization
-Partial or total replacement by a
Software Dev. Activities
Maintenance - continued
• Address problem and modification
analysis
• Address modification implementation
• Address maintenance
review/acceptance
• Objectives: to assess proposed
changes
and their impact on the software,
evaluate
anomalies that are discovered during
operation, assess migration
Software Dev. Activities
Maintenance – Unique Tasks
• Software V & V Plan Revision(s)
• Proposed Change Assessment
• Anomaly Evaluation
• Migration Assessment
• Retirement Assessment
• Task Iteration
V & V Manager Role
Acquisition V & V
• Necessary if your project includes
the
purchase or contract development
software
• The V & V Plan must include tasks to

address the acquisition and

integration of
that software into your environment
Acquisition V & V
Acquisition Activities
• Address project initiation
• Address request for proposal
• Address contract preparation
• Address supplier monitoring
• Address acceptance and completion
• Objectives: to scope the V & V effort,
plan
interfaces with the supplier and
acquirer,
and review the draft systems
Acquisition V & V
Acquisition Tasks
• Scoping the V & V effort
• Planning the Interface between the V
&V
Effort and the Supplier
• System Requirements Review
V & V Manager Role
Supply V & V
• Necessary if you are developing
custom
applications or components for an
external
customer
• Initiated by a decision to prepare a
response to a request for proposal
• Initiated by signing and entering into
a
contract to provided the system,
software
V & V Manager Role
Supply V & V - continued
• Determination of procedures and
resources
required
• Verify that the request for proposal
requirements and contract
requirements
are consistent with customer needs
• Uses the contract requirements and
program schedules to revise and
update
the interface planning between the
Supply V & V
Supply Activities
• Addresses the initiation
• Addresses the preparation of response
• Addresses contract planning
• Addresses execution and control
• Addresses review and evaluation
• Addresses delivery and completion
activities
Supply V & V
Supply Tasks
• Planning the Interface between the V
&V
Effort and Supplier
• Contract Verification
V & V Manager Role
Reporting
• Consists of task reports, activity
reports,
anomaly reports and the final report
• Reports are provided as feedback to
the
software development process
regarding
the technical quality of each software
product.
Reporting
Base-set of V & V reports
• Task Reports
• V & V Activity Summary Reports
• Anomaly Report
• V & V Final Report
• Optional Reports (example: special
study)
V & V Summary
Emphasis on Verification
Concepts - definition
Techniques
Stressed Reviews & Inspections – types &
classes
In-Process
Milestone
Test Readiness
Test Completion
Post-Implementation
Informal
Semi-formal
Formal
V & V Summary
Emphasis on Verification
Roles for Verification & Validation
Focus on V & V Manager - activities
Software Development
Acquisition
Supply
Reporting
V & V Summary
Things to Remember
The earlier in the software development
cycle, defects are detected, the easier and
cheaper they are to fix.
Over 65 % of defects are found in
Verification Reviews
Less than 30 % of defects
are found in the testing
phase
V & V Summary
Things to Remember
The product is reviewed, not the producer
Defects and issues are identified, not
corrected
All team members of the review are
responsible for the results of the review
V & V Summary
Mentioned Validation
Concepts – definition
(provided by an outside source)

Mentioned Audits
Concepts - definition