Safety

Critical
software
[Software Testing &
tools]

M S Prasad

This lecture note is based on open literature and information available on web. It is
suitable for Grad/Post grad students of Avionics.
 Testing safety-critical software systems
And Tools :
Introduction

A safety-critical software system could be defined as any system whose failure

or malfunction can severely harm people's lives, environment or equipment.
These kinds of risks are managed using techniques of safety engineering.

Safety-critical systems are widely used in various different fields such as

medicine, nuclear engineering, transport, aviation, aerospace, civil
engineering, industrial simulation, process control, military devices,
telecommunications, infrastructures, etc.

Safety-critical systems consist of hardware equipment and software and both

of them have to be secure in order to ensure that the whole system is fully
secure.

“There are three basic approaches to achieving reliability [of safety-critical

systems] – 1) Testing – lots of it; 2) Software fault tolerance; and
3) Fault avoidance – by formal specification and verification, automatic
program synthesis and reusable modules” [Lawren 92].

. Safety-critical standards

Industries have developed a number of different standards to deal with the

development of safety-critical systems. For instance, we can find standards in
medicine, to develop medical device software, in nuclear engineering, to
develop nuclear power control stations software, in transport, to develop
software for railway control, in aviation, etc. In general, every field which uses
safety-critical systems has its own standards to handle the development of the
systems they need .

Safety Critical Software Testing # m s prasad Page 1

Standards for safety critical software have been classified on a scale of 5 levels
of safety integrity: 4 is very high, while 0 is a system which not safety related.
Formal mathematical methods are recommended by most standards at
integrity level of 4 whereas not required at levels 0 or 1. Standards are legal
and regulatory requirements help in the design of safety-critical software
systems as they make the designers follow the requirements.

DO‐178B ‐ Software Considerations in Airborne Systems and Equipment

Certification
Standard of RTCA Incorporation (in Europe it is ED‐ 12B and standard of
EUROCAE). This represents the avionics industry consensus to ensure
software safety. Acceptable by FAA and EASA certification authorities.
1. DO‐178 in 1982
Basic concepts of SW design assurance
Three levels of SW safety definition
2. DO‐178A in 1985
Concentrates on testing and configuration management
. DO‐178B in 1992
Five levels of SW safety
From Testing focus to requirement‐based
DO‐278 in 2002
Interprets DO‐178B to ground and space based‐systems
DO‐178C Incorporates modern SW development and analysis techniques
DO-178B requires that all system requirements be mapped to one of the five
software levels.
Level A - most critical failure level. Failure at Level A results in catastrophic
failure conditions for an aircraft.
Level B - the software at this level contributes to severe-major failure
conditions.
Level C - software at this level contributes to major failure conditions.
Level D - software at this level contributes to minor failure conditions.

Safety Critical Software Testing # m s prasad Page 2

Level E - the software has no effect toward potential failure conditions.
Different failure conditions require different software conditions as shown
below :

Some examples of safety critical System Level A and B

Fly By wire controls ;Auto pilots ,RADAR , FADEC system ;IFF
Missile systems ,Collision avoidance system
Safety Critical C and D
Mission Planning and implementation ,Telemetry , Health Monitoring etc

The Avionics software standard DO178 B defines four classes of software :

-
A. User modifiable Software
B. Option Selectable software

Safety Critical Software Testing # m s prasad Page 3

C. Commercial Off The self Software
D. Field Loadable software
Accordingly to maintain the quality it defines four building blocks for
software : -
a. Requirement Definition
b. Design the program
c. Code the program
d. Integrate the program

Programming features and languages

When designing a safety-critical software system, the general principle applied
is to try to keep the system as simple as possible. This principle, of course,
extends to the decision of which programming language is going to be used to
code the system.
In most modern programming languages there are some particular features
which could cause problems and therefore should be avoided for safety-
critical software systems. It is recommended to avoid the following aspects
during software development :

 Pointers and dynamic memory allocation/de allocation.

 Unstructured programming: use of go to .
 Variant data.
 Multiple entry and exit points to loops, blocks, procedures and functions.
 Implicit declaration and implicit initialisation.
 Procedural parameters.
 Recursion.
 Concurrency and interrupts.

On the other hand, other programming features which provide reliability and
are less likely to lead to errors are:

 Strong typing.
 Runtime constraint checking.
 Parameter checking.
 And, in general, programming languages well structured in blocks and
Safety Critical Software Testing # m s prasad Page 4
 which force modular programming.

Among the programming languages with the good features required

mentioned above, Ada stands out as one of the most reliable and secure,
however, as none programming language is perfect, the common approach is
to use a small subset of it in order to avoid the risky features and make the
most of the reliable ones. The Ada subset most commonly used for safety-
critical software systems is called SPARK.

Approaches on designing safety-critical software systems

Although there are many different approaches on designing safety-critical
software systems, the two main approaches are described below.
The basic idea when designing safety-critical software systems is to identify
hazards as early as possible in the development life-cycle and try to reduce
them as much as possible to an acceptable level.
The first approach considered is to formally prove that the system does not
contain errors by construction by means of formal methods. Formal methods
are mathematical techniques and tools used to specify, design and verify
software systems. Specifications are written as well-formed statements using
logic mathematical language and formal proves are logic deductions using
rules of inference.

However, using formal methods for large scale systems is fairly complicated
and time-consuming and has big problems such as human introducing errors
in the specification and proves.
It is also almost impossible to formally prove everything used to develop the
system such as the compiler, the operating system in which the system will
ultimatelyoperate..
For small systems, where formal specifications and proves are easier to deal
with, the approach can be very successful. The technique used to overcome
problems with large scale systems is to try to separate the critical functionality
of the system from the other non-critical parts. This way of using components
with different safety integrity levels works well providing it is proved that the
non-critical components cannot affect the high integrity ones or the whole
system.

Note :
. “The program used to control the NASA space shuttle is a significant example
Safety Critical Software Testing # m s prasad Page 5
of software whose development has been based on specifications and formal
methods. As of March 1997 the program was some 420,000 lines long. The
specifications for all parts of the program filled some 40,000 pages. To
implement a change in the navigation software involving less than 2% of the
code, some 2,500 pages of specifications were produced before a single line of
code was changed. Their approach has been outstandingly successful. The
developers found 85% of all coding errors before formal testing began, and
99.9% before delivery of the program to NASA. Only one defect has been
discovered in each of the last three versions. In the last 11 versions of the
program, the total defect count is only 17, an average of fewer than 0.004
defects per 1,000 lines of code.”

The second approach is based on assuming that errors exist and the main aim
will be to design prevention and recovery mechanisms in order to avoid
hazards or risks caused by the system. These mechanisms will go from small
parts of the code such as control errors inside procedures and functions,
through all the software and ending on the whole system. This prevention
and recovery mechanisms are based sometimes on redundancy approaches
which mean replicating the critical parts of the system. For instance, it is
common to use redundancy techniques systems in aircrafts where some parts
of the control system may be triplicated. An example of redundancy focusing
only in the software part of a critical-system is the N-version programming
technique also known as multi-version programming. In this approach,
separate groups develop independent versions of the same system
specifications. Then, the outputs are tested to check that they match in the
different versions. However, this is not infallible as the errors could have been
introduced in the development of the specifications and also because different
versions may coincide in errors.

Testing safety-critical software systems

In the process of verifying safety-critical software systems, the same

techniques applied to typical software systems are still used, as well as special
techniques conceived to minimise risks and hazards. The techniques are
obviously applied in a more formal and rigorous way than in typical systems.
Independent verification is usually required in those systems by means of a
separate team within the overall project structure or a verification team
supplied by an external company who may not ever meet the development
Safety Critical Software Testing # m s prasad Page 6
team, depending on the criticality of the system.

We have to remember that when testing software at different stages of its

development, tests are always performed to verify correct behaviour against
specifications, not against observed behaviour. For this reason, design of test
cases for coding, should be done before coding the software system.
Otherwise, software developers are tempted to design test cases for the
behaviour of the system which they already known, rather than for the
specified behaviour.

Some well-known techniques used to generate test cases to test these kinds of
systems are white box and black box testing and reviews. However, they are
taken to a further level of detail than with typical systems.
Complex static analysis techniques with control and data flow analysis as well
as checking that the source code is consistent with a formal mathematical
specification are also used. Tools such as SPARK Examiner are available for that
[12]. Dynamic analysis testing and dynamic coverage analysis are also
performed using known techniques such as equivalence partitioning,
boundary value analysis and structural testing.
Any tools used to verify and test safety-critical software systems have to be
developed in the same formal way as the systems they will test
Considering now some of the specific techniques from safety engineering to
test and verify safety-critical software systems, we can name a range of them
such as probabilistic risk assessment, a method which combines failure modes
and effect analysis (FMEA) with fault trees analysis (FTA), also failure modes,
effects and criticality analysis (FMECA), an extension of the former FMEA,
hazard and operability analysis (HAZOP), hazard and risk analysis and finally
tools such as cause and effect diagrams (also known as fishbone diagrams).
The main idea behind all these techniques is the same. The first step in PRA is
to perform a preliminary hazard analysis to determine which hazards can
affect system safety. Then, the severity of each possible adverse consequence
is calculated. It can be classified as catastrophic, hazardous, major, minor or
not safety related. The probability of each possible consequence to occur is
also calculated next. We can classify them in probable, remote, extremely
remote and extremely improbable. Assessment of risks is made by combining
both the severity of consequences with the probability of occurrence (in a
matrix). For this evaluation we use different risk criteria like risk-cost trade-
offs, risk benefit of technological options, etc. Risks that fall into the
Safety Critical Software Testing # m s prasad Page 7
unacceptable category (e.g.: high severity and high probability), that is to say,
are unacceptable, must be mitigated by some means such as safeguards,
redundancy, prevention and recovery mechanisms, etc., to reduce the level of
safety risk. Probabilistic risk assessment also uses tools such as cause and
effect diagrams .

Failure modes and effect analysis (FMEA) is a procedure for analysis of

potential failures within a system for classification by severity or
determination of the effect of these failures on the system. Failure modes can
be defined as any errors or defects in a process, design or item, especially
those that affect the customer and can be potential or actual. Effects analysis
refers to studying the consequences of these failures. Failure modes, effects
and criticality analysis (FMECA) is an extension to this procedure, which
includes criticality analysis used to chart the probability of failures against the
severity of their consequences.
Fault trees analysis is a graphical technique that provides a systematic
description of the combinations of possible occurrences in a system which can
result in an undesirable outcome (failure). An undesired effect is taken as the
root of a tree of logic. Each situation that could cause that effect is added to
the tree as a series of logic expressions. Events are labelled with actual
numbers about failure probabilities. The probability of the top level event can
be determined using mathematical techniques. Here, we can see an example
of a FTA:

Safety Critical Software Testing # m s prasad Page 8

--------------------------------------------------------------------------

. References

[1] Wikipedia, Life-critical system or safety-critical system definition.

Available at: http://en.wikipedia.org/wiki/Life-critical_system

[2] Software Engineering Process Technology (SEPT), Software Safety

Standards.

[3] Bill St. Clair, Software securing critical systems, VME and Critical Systems,
2006,
[4] IPL Information Processing Ltd, An Introduction to Safety Critical Systems,
Testing Papers.
[5] NASA LaRC Formal Methods Program: What is Formal Methods?

[6] Frederick P. Brooks, Jr. , No Silver Bullet: Essence and Accidents of Software
Engineering,
1986.

[7] R.D. Tennet, Specifying Software: A Hands-On Introduction, 2002.

[8] Wikipedia, Redundancy (Engineering).

---------------------------------------------------------------------------------------------------------

Safety Critical Software Testing # m s prasad Page 9

 There are many software testing tools available for testing general purpose software
and carrying out Static or Dynamic analysis with defined software Metrics . In this lecture
note we are going to restrict our discussion to THREE software testing tools which are
widely used :
a. IBM Rational DOORS
b. LDRA testing tools
c. Canata C++ testing tools .

IBM Rational DOORS

IBM Rational DOORS® software is designed to capture, trace, analyze and manage
changes to requirements and helps to comply with industry standards. It Supports
collaborative, intuitive and scalable management of requirement specifications..
It also provides automatic notification of changes that can impact related requirements,
designs and tests.
Rational DOORS provides:
 Requirements management in a centralized location for better team collaboration.
 Traceability by linking requirements to design items, test plans, test cases and other
requirements.
 Scalability to address your changing requirements management needs.
 Test tracking toolkit for manual test environments to link requirements to test
cases.
 Integrations to help manage changes to requirements with either a simple pre-
defined change proposal system or a more thorough, customizable change control
workflow with Rational change management solutions.

LDRA

Requirements Traceability Tools from LDRA

 TBmanager is a role-based requirements management tool that lets teams work on
individual activities and link code and verification artifacts back to higher-level objectives.
 TBreq provides traceability throughout the workflow and quickly identifies broken
links.
 TBeXtreme is an optional module that detects code changes and determines
software regression testing requirements.
 The LDRA tool suite is ideal for development teams who are moving to a more
formal development process to improve code quality and maintainability, including static
and dynamic analysis, coding standards compliance, and unit, integration, and system
testing.
Safety Critical Software Testing # m s prasad Page 10
 LDRA provides a comprehensive, role-based approach to traceability
Requirements and verification tasks can be assigned to team members, and all resulting
artifacts can be aggregated and linked. The result is a completely traceable process
throughout the software development life cycle so you can ensure that any changes to
requirements, design, or source code are easily understood, verified, and traced.

Coding Standards Compliance Tools from LDRA

 LDRA Testbed/TBvision is the static analysis engine and code visualisation
capability that helps enforce compliance with coding standards and clearly indicates
software flaws that might otherwise pass through the standard build and test process
 LDRArules is a cost-effective, stand-alone rules checker that doesn’t require
investment in the complete tool chain
LDRA automates static analysis and rules checking with tools that support almost any
standard—With LDRA’s code visualisation, you can identify exactly where the source code
deviates from the standard so it can be rapidly addressed. While many rules checking
tools are created equally (because they’re based on the same commercial third-party
parsing engine), LDRA’s standards compliance tools offer significant advantages. We have
complete control over our parsing technology, which means we can quickly adapt to
variations in languages and language constructs. And with our long involvement in
standards organisations such as MISRA, we’re able to respond rapidly with updated rules
checking as new standards are released.

Our coding standards compliance tools let you combine standards and define appropriate
rule subsets, select individual rules, and add your own. Within the tool, you can easily
check for coding standards compliance to any single standard or combination of
standards or subsets. And you can check compliance of a single code base against
multiple standards to compare how the code fulfills each one and see what would be
required to adapt the code to conform to it.

Code Coverage Analysis Tools from LDRA

 TBrun with the TBextreme module provides automated unit testing, test case
generation and execution, and code coverage analysis
 The TBobjectbox module adds object code verification
 The Tool Qualification Support Pack (TQSP) provides documentation on our own
tools and qualification guidance and artifacts to meet certification requirements
 LDRAcover is a code coverage stand-alone tool to help meet structural coverage
analysis objectives

Safety Critical Software Testing # m s prasad Page 11

 LDRAunit is a stand-alone tool for automated unit test generation and
management

LDRA’s coverage analysis tools address the most stringent requirements in the safety-
and security-critical markets
We are the only company that offers coverage analysis at both the source code and
object code levels to help you meet the most stringent coverage requirements. LDRA
tools automatically generate test cases, execute those test cases, and visually report
levels of coverage analysis, such as statement, branch/decision, procedure/function call,
MC/DC, dynamic data flow, and more.

Our tools support C, C++, Java, Ada and Assemblers, running on a broad range of target
platforms—from powerful 64-bit microprocessors to highly constrained 8- or 16-bit
microcontrollers. LDRA tools can automatically generate test cases that provide 50-80% of
coverage. And our intuitive test case building environment lets developers quickly
augment those test cases to increase their coverage if necessary.

LDRA is a stand alone tool to meet Code coverage a sper DOD standards.

Unit/Integration & Target Testing Tools from LDRA

 TBrun provides automated unit test and system integration testing capabilities. It is
used after dynamic analysis to achieve full MCDC , Statement and branch analysis.
 LDRAunit is a stand-alone tool that provides a complete integrated environment for
automated unit test generation and management, without requiring investment in a
complete tool chain
 TBextreme (an optional module for either TBrun or LDRAunit) revolutionises unit
testing by providing a totally automated solution for test vector generation

LDRA lets you perform unit and integration tests on the host and target hardware
With LDRA, you can quickly and easily generate and execute tests at the unit and
integration levels, both on the host (standalone or with target simulation) as well as on
the target hardware. We provide test generation (test harness, test vectors, code stubs)
and result-capture support for a wide range of host and target platforms. Our optimised
instrumentation technology lets you pull test information even off highly constrained 8-
and 16-bit microcontrollers, and through high-performance 32- and 64-bit processors.

With that range of support, your team has a common unit testing and integration testing
environment for multiple projects with different target platforms.

Safety Critical Software Testing # m s prasad Page 12

Safety Critical Software Testing # m s prasad Page 13