Reliability Engineering and System Safety 91 (2006) 882–893

www.elsevier.com/locate/ress

Reliability demonstration test planning:

A three dimensional consideration
Om Prakash Yadav a,*, Nanua Singh b, Parveen S. Goel c
a
Department of Industrial and Manufacturing Engineering, North Dakota State University, Fargo, ND 58105, USA
b
Department of Industrial and Manufacturing Engineering, Wayne State University, Detroit, MI 48202, USA
c
TRW Automotive, Chassis System, EAS, Sterling Heights, MI, USA
Received 10 February 2005; received in revised form 25 August 2005
Available online 2 November 2005

Abstract
Increasing customer demand for reliability, fierce market competition on time-to-market and cost, and highly reliable products are making
reliability testing more challenging task. This paper presents a systematic approach for identifying critical elements (subsystems and components)
of the system and deciding the types of test to be performed to demonstrate reliability. It decomposes the system into three dimensions, (i.e.
physical, functional and time) and identifies critical elements in the design by allocating system level reliability to each candidate. The
decomposition of system level reliability is achieved by using criticality index. The numerical value of criticality index for each candidate is
derived based on the information available from failure mode and effects analysis (FMEA) document or warranty data from a prior system. It
makes use of this information to develop reliability demonstration test plan for the identified (critical) failure mechanisms and physical elements.
It also highlights the benefits of using prior information in order to locate critical spots in the design and in subsequent development of test plans.
A case example is presented to demonstrate the proposed approach.
q 2005 Elsevier Ltd. All rights reserved.

Keywords: Reliability demonstration; Product development; Reliability improvement

1. Introduction What is really needed is a comprehensive approach, which

The problem of reliability demonstration of a new product
using existing test methods is well known in manufacturing
industry, especially in automotive industry. Much of the testing
that is performed in industry is based upon traditions,
standards, and procedures that do not provide the optimum
balance of assurance versus cost and time [1]. Increasing
product complexity, fierce market competition on time-to-
market and cost, and highly reliable products are making
reliability demonstration even more challenging task. It
necessitates that product development (PD) process should be
supported by more effective design validation and verification
program to ensure that reliability requirements are being met
by the new design. Furthermore, reliability assessments
performed during the PD process should also be utilized
effectively in identifying potential faults and weaknesses in a
design during the critical planning and development phase.
* Corresponding author. Tel.: C1 701 231 7285; fax: C1 701 231 7195.
E-mail address: om.yadav@ndsu.edu (O.P. Yadav).
0951-8320/$ - see front matter q 2005 Elsevier Ltd. All rights reserved.
doi:10.1016/j.ress.2005.09.001
proactively integrates reliability demonstration into the PD
process, provides a systematic procedure as support tool to
development process, and helps identify critical links (or weak
spots) in the design.
Many people have attempted to develop design verification
and testing plan [2–6] by proposing frameworks for sample
size determination. Porter [7] suggests an accelerated
reliability qualification method in automotive testing based
on failure mode verification test (FMVT). With the FMVT, the
testing is conducted on a single sample by applying the stress
sources simultaneously starting at service conditions and
increasing to a predetermined maximum test level. Gerling,
Preussger, and Wulfert [8] have attempted to address the
problem of reliability qualification of semiconductor devices
based on physics-of-failure and risk and opportunity assess-
ment. There have been some efforts to consider failure
mechanisms (competing risks) for each component to predict
reliability [9–11]. The existing models are capable of providing
the sample size requirements for a specified system reliability
target and confidence level. However, some of the very
important questions remain unanswered as to what types (or
categories) of test to perform, how many of each, and which
 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893 883

component or sub-system is subjected to each test category.
How do we utilize reliability assessments to identify weak
spots or links in the design if there is no failure during testing?
To answer these questions, one requires a thorough under-
standing of system concept (or structure), failure behavior,
criticality of each element of system, and a mechanism to
identify the weak spots in the design, which may need further
analyses, design changes, and/or testing. In order to develop
more effective testing plans, there needs to be a transition to a
science-based approach by understanding the physical,
functional, and damage behavior—time dependent degradation
process or failure mechanism—aspects of the system.
The successful development and demonstration of
reliability requirements requires hierarchical analysis of
physical and functional dimensions, which enables one to
see, in detail, how functional requirements can be fulfilled by
each individual element and which failure mechanism might
affect their capability to do so. Therefore, the primary goal of
any PD process is to design the physical structure of the final
system or product, which is capable of fulfilling all functional
requirements of the customer over a specified period of time.
This time-dependent aspect of functional requirement is known
as reliability. This three dimensional understanding of a system
shows that each dimension is equally important. Fig. 1
illustrates three dimensions of a system design.
The consideration of three dimensional analyses in design
and development process serves as the basis for answering
questions which were raised earlier. These questions can be
answered by understanding physical relationship between
various elements of the system, their role in performing
different functions, and identifying potential failure mechan-
isms affecting each element and function. The following
paragraphs give a brief understanding of importance and
suitability of this approach in formulating reliability demon-
stration strategy based on three dimensional thinking or concept.
The three dimensional analysis decomposes product
structure into its physical elements enabling one to see, in
detail, how functional requirements can be fulfilled by
individual product element or by a group of elements. It also
allows one to trace properties of an element which are essential
to perform certain functional requirements. In principle, the
functional requirements also propagate from the requirements
for the complete product down to the elements in a hierarchical
manner. This hierarchical decomposition of product and
functional requirements will help answering the following
question:
Which physical element(s) is responsible for the fulfillment
of a specific functional requirement?
The mapping between physical elements and functional
requirements will show up, which physical elements have
impact on the same function or one element has an impact on
different functions. This information provides answer to the
question; which component or sub-system to test?
For time dimension (or damage behavior), the under-
standing of conditions of operation and time in application
plays an important role. The knowledge of failure mechanism
and results of failure analysis will help in identifying which
failure mechanisms have impact on the same physical element
or one failure mechanism has impact on various physical
elements. This understanding will answer one of the important
questions: what types of test to perform?
In order to answer the questions regarding the sample size
and duration of each test, we propose to use the existing
knowledge and information such as warranty data and impact
of corrective actions and design changes to assess the current
estimates of reliability parameters and hence determine test
duration and sample size [12]. The difference between current
reliability estimate and reliability target might influence the
decision of determining the amount of test duration or sample
size requirement for reliability demonstration. The effective
correlation and mapping between three dimensions, while
planning reliability demonstration strategy, ensures that if
reliability targets can be achieved, then functional require-
ments will also be achieved [8].
This paper proposes a comprehensive framework, which
facilitates the development of reliability test plan by bringing

Physical Structure

ct
o du
pr s
ce
al tem rvi
Fin -sy
s e
b ofs it ife
/ su g lim dl
ies nin nty ign
e
mb
l gin ar
ra s
se be W De
-a
s nts or
b e w
Su p on Ne Time in Service
C om (Damage behavior)
Component level functions

Sub-system level functions

System level function

Functional Requirements

Fig. 1. Three dimensions of product design.

884 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893
three dimensional understanding of the product design while
utilizing existing information and knowledge generated during
PD process. To capture three dimensional understanding, the
system is decomposed along the three dimensions, i.e. physical
structure, functions, and time (or damage behavior). Since the
damage behavior of any system aggravates with usage time,
therefore, the time dimension of product design is captured
through damage behavior or failure mechanism of the system.
Fuzzy logic based reliability improvement assessment method
[12] is used to assess system level reliability utilizing available
evidence both quantitative and qualitative. The estimated
system level reliability and reliability target are allocated to
each element of the decomposed system using a criticality
index. The criticality index for each element is calculated
utilizing criticality numbers or risk priority numbers (RPN)
from FMEA document or information available from other
sources such as warranty data or field failure data from a prior
similar product. Reliability demonstration test plan is
formulated if the current reliability estimates for any candidate
(or element) are less than their allocated reliability targets. The
framework allows continuous updating of system reliability as
new evidence is available at any stage of the PD process. The
Weibull distribution is considered as failure time distribution
with assumption of known shape factor value. The Bayesian
framework is used to update the reliability estimates based on
test results and any other new evidence. For more detail on this
refer [12,13].
The paper is organized as follows. Section 2 describes the
proposed methodology of reliability demonstration test
planning. Section 3 discusses three dimensional decomposition
of the system. Section 4 presents criticality index concept and
reliability allocation methods. Section 5 discusses system
reliability assessment model and test plan to demonstrate
reliability goal. Section 6 presents a case example to
demonstrate the proposed methodology. And Section 7
presents some concluding remarks.
2. The proposed methodology
In the product development process each stage is
considered as an interim milestone to assess the product
reliability and make further decisions for additional testing
and resource assignment. At each stage different types of
tests are required to be performed to demonstrate product
reliability and durability and of course to identify and
eliminate potential failure modes. The proposed method-
ology of design verification and reliability demonstration
planning is an attempt to help design community to develop
more cost effective and robust test plan. The purpose of the
proposed methodology is to ensure that system reliability as
well as components/sub-systems reliability estimates are
meeting the reliability targets or goals set by the customer
at each stage of the product development process. The
proposed framework helps identify types of test to be
conducted to demonstrate the reliability targets, sample size,
test duration, and components (or sub-systems) to be tested.
Fig. 2 illustrates the flow chart for various activities/steps in
the proposed methodology.
We propose the following six generic steps for developing
the comprehensive design verification and reliability demon-
stration (DV&RD) planning model advocated here. The
proposed framework is completed in steps 1–6.
1. Identify the system for which the reliability demonstration
testing is to be performed.
2. Conduct product structure and failure analysis to decom-
pose the system into its three dimensions, i.e. physical
structure, functional requirements, and time dimension (or
damage behavior). Create two-dimensional and three-
dimensional matrices to identify the interfaces between
sub-systems (or components), functions, and failure
mechanisms.
3. Allocate the target reliability to each (appropriate) interface
or cell and to each sub-system, function, and failure mode
using Eqs. (1) – (4).
4. Allocate prior reliability information to each sub-system,
function, and failure mechanism and also to each
appropriate interface (cell) of the matrices and develop
prior distribution parameters.
5. Update the appropriate reliability components (rijk,rij,ri,rj or
Rs, for detailed nomenclature refer to Section 5) based on
available information, which could be both qualitative, as
well as quantitative. To incorporate qualitative information,
use the fuzzy logic based reliability improvement assess-
ment approach.
6. Compare the estimated reliability with reliability targets for
each element. If the reliability target is achieved, then stop;
otherwise, develop reliability demonstration test plan to
demonstrate the reliability target for any component or
failure mechanism using the Eqs. (14) – (19), conduct tests
to collect more information and go back to step 5. If test
failed to demonstrate the required reliability targets, make
necessary design changes and go back to step 5 to update
reliability estimates using fuzzy logic model approach.
3. System structure and failure analysis
System structure and failure analysis is a process that results
into three-dimensional decomposition of a product, i.e. its basic
functional and physical elements along with potential failure
mechanisms (competing risks) for each functional and physical
(component/sub-system) interface. Functional elements are
defined as the individual operations and transformations that
contribute to the overall performance of the product. Physical
elements are defined as the sub-systems or components that
ultimately implement the system’s function. Failure mechan-
ism is defined as the process or damage behavior by which a
component’s ability to perform a required function is
terminated. The system structure analysis consists of system
physical decomposition and system functional decomposition.
The system failure analysis results in identification of different
 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893 885

Step 1
Select a system

Step 2
Product and functional Decompose system into its three
FMEA analysis
structure diagram Dimensions and make 2&3
dimensional matrix

Step 3
Customer requirements
Allocate reliability targets

Step 4
Failure analysis,
Allocate prior reliability & develop
warranty information etc.
Prior distribution

Step 5
Quantitative information Qualitative Information
Update reliability estimates on
(test results) (fuzzy logic model output)
current product

Yes Step 6 Yes

Develop test plan and Make necessary design
Is
Conduct additional tests Failed to Changes or modifications
R<R*?
Demonstrate R*

No

STOP

Fig. 2. Flow chart for reliability demonstrations strategy.

failure mechanisms (competing risks) contributing to the 3.2. Functional decomposition

failure of a component to perform a particular function.
3.1. Physical decomposition
In physical decomposition, the system is decomposed into
its basic physical components which, when assembled
together, will accomplish the system/product function. The
physical decomposition should result in identification of basic
components that must be designed to perform the system
function at a required reliability level. The decomposition
process should continue until basic physical components are
reached.
System functional decomposition describes the systems
overall functions and identifies component functions. In
functional decomposition the overall function is broken into
sub-functions, which are further decomposed into lower-level
functions. This functional breakdown continues until a set of
functions that could be achieved by the available components
is reached.
At this point all functions are mapped into components.
Components are arranged into sub-systems leading to an
overall system that will accomplish the overall function. Fig. 3
illustrates the hierarchical mapping matrix of all functions and

System
Sub-system 1 … Sub-system k
C11 … C1n … Ck1 … Ckn
Sub-Function 1
Sub-function 2
Function

.
.
.
Sub-function q

Fig. 3. Functional-component mapping matrix.

886 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893

Failure Modes (k)

Sub-function (i) Component (j) 1 … m
1
…
…
1 p
1
…
…
… p
1
…
…
q p

Fig. 4. Three-dimensional mapping matrix.

components into overall function and system. The black cells system level reliability. Let R
s be the required system


indicate that a particular component(s) or sub-system is not reliability and rijk be the required reliability for ith function,
contributing to the corresponding function. jth component, and kth failure mechanism (each interface or
cell) in the three-dimensional matrix. Then
3.3. Failure analysis n

1=p o1=q 1=m


rijk Z R
s ðtÞ (1)
The failure analysis is a systematic approach for analyzing
effects of a failed component on a particular function and
overall system function. Effective analyses help identify a where iZ1,2,.,p jZ1,2,.,q and kZ1,2,.,m
failure mechanism or mode and show how it causes termination The required reliability for each element can be given as:
of a particular function. The failure analysis exercise continues
1=p
until one identifies all potential failure mechanisms for ri
Z R
s ðtÞ
different functional and component interfaces. The mapping

1=q
of failure mechanisms adds a third dimension to the previously rj
Z R
s ðtÞ
defined functional and component mapping matrix as given in
Fig. 4. The black cell(s) indicate that a particular functional and and
component interface is not affected by the corresponding
1=m
rk
Z R
s ðtÞ (2)
failure mechanism.
To consider the complexity of any element in reliability
4. Reliability allocation/decomposition allocation or the relationship between elements and system
failure, we propose the following reliability allocation model.
The process of assigning reliability requirements to If Rs(t) is the estimated system reliability, the allocated
individual components (sub-systems) to attain the specified reliability estimate for any element or interface between all
system reliability is called reliability allocation. The objective elements can be given as:
of reliability allocation is to use the reliability model to assign
reliability targets to each sub-system, functions, and failure rijk Z 1Kb1KfRs ðtÞgwijk c
mechanisms (competing risks) so as to achieve a specified
reliability goal for the system. Reliability allocation methods ri Z 1Kb1KfRs ðtÞgwi c
can also be used to estimate components, functions, and failure
mechanisms reliabilities when system-level test results or rj Z 1Kb1KfRs ðtÞgwj c
failure data are available. For the sake of differentiation,
allocation of system reliability estimate to different elements/
rk Z 1Kb1KfRs ðtÞgwk c (3)
dimensions of system is termed as reliability decomposition.
Depending on the degree of complexity associated with X X X XXX
wi Z wj Z wk Z wijk Z 1
each element (components, functions, and failure mechan-
i j k i j k
isms), the appropriate reliability allocation method can be
selected as suggested by Kapur and Lamberson [14]. The equal where wi, wj, wk, and wijk are defined as criticality index of
apportionment technique can be used to assign equal functions, components, failure mechanisms, and their inter-
reliabilities to all elements in order to achieve a specified faces, respectively.
 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893 887

The transformed failure rate [12] allocation for any element normalizing the criticality number so that the terms add up to 1:
is given as: XXX
  wijk Z 1
wi KlnRs ðtÞ i j k
lti Z
tb To use the system-level test results from the current design,
  if the cause/source of failure is reported for each system failure
wj KlnRs ðtÞ then it can be used to compute the criticality index to
ltj Z
tb decompose system reliability estimate. For example, let n be
  the total number of system failures and ni be the number of
wk KlnRs ðtÞ system failures caused by ith component or failure mechanism
ltk Z seen during the system-level testing. Then the criticality index
tb
for each component or failure mechanism can be computed as:
and X
n
  wi Z i where wi Z 1: (5)
wijk KlnRs ðtÞ n i
ltijk Z (4)
tb
For any element if niZ0 then allocate very small value to
where tZMission time for reliability calculation and bZShape criticality index, .01 or .001 rather than zero to avoid allocating
factor for Weibull distribution. reliability riZ1, which is somewhat unrealistic.

4.1. Criticality index 5. System reliability assessment model

The criticality index of any component, function, failure
mechanism, and their interface explains the complexity of each
one of them in the system and is defined as contribution of each
to the system failure. The numerical value of criticality index
for each candidate is calculated based on the information
available from different sources such as warranty data or field
failure data from a prior system, FMECA analysis, and test
results of the current system.
The failure analysis of the warranty or field failure data is
the most desirable and appropriate source of information to
determine the criticality index because it captures the field use

behavior of the system. The relative frequency distribution of
source/cause of failure is used to compute the criticality index.
The relative frequency of system failure assign to each failure
source/cause is treated as criticality index. Fig. 5 illustrates the
relative frequency of system failure for different failure
mechanisms.
The information generated during FMECA analysis [15] is
also useful and can be used to compute the criticality index for
each interface between components, functions, and failure
mechanisms. The FMECA process assigns a criticality number
(or RPN) to each combination of component, function, and
failure mechanism. The criticality index is computed by
Corrosion
15%
Fatigue
40%
Sterngth
20%
Fatigue
Wear
Sterngth
Wear
25% Corrosion
Fig. 5. Failure distribution of warranty data.
5.1. Failure time distribution
The Weibull distribution is often used for the product life
because it describes increasing and decreasing failure rates. It
may be suitable for a ‘weakest link’ product; that is, the product
failure can be attributed to many parts/competing risks with
comparable life distribution and the product fails with the first
part failure or competing risk occurrence. Therefore, the time-
to-failure of each failure mode/function/component is assumed
to follow the Weibull distribution
  
b t b
f ðtÞ Z b tbK1 exp K (6)
q q
where b is shape factor, and q is characteristic life or scale
factor.
Here we assume that shape factor b is known and has same
value for all components and failure modes. Another
underlying assumption is that each failure mechanism/compet-
ing risk occurs independently.
Soland [16] shows that the transformed failure time TZtb
follows exponential distribution with constant transformed
failure rate ltZ1/qb and mean time to failure is given as qb.
Under the assumption that b is known, the gamma distribution
used in a standard Bayesian framework as prior distribution for
exponential distribution can then be easily applied to derive the
posterior parameters for transformed failure rate for failure
data following the Weibull distribution.
5.2. System reliability
Most of the mechanical systems are designed in a series
system. The current system under consideration is also
assumed to have a series configuration. Then the reliability
of the series system [17], assuming that the failure of any
function, component, and failure mode combinations
888 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893

(interfaces) will lead to system failure as if they are connected updating of product reliability as more and more information
in series, could be written as: is gathered. The probability density function of gamma
" ( )# distribution is given as:
Yp Yq Ym
Rs ðtÞ Z Rijk ðtÞ ba00 la0K1 expðKb0 lÞ
iZ1 jZ1 kZ1 gðlÞ Z (12)
Gða0 Þ
or

 Where a0 and b0 are the two parameters of the prior
Rs ðtÞ Z exp Klts tb (7) distribution. For all elements of the decomposed system, (i.e.
components, functions, and failure mechanisms or competing
Reliability of kth failure mode for ith function and jth risks as well as system), these two parameters are estimated
component for mission time t (h/mile) is given as: based on the available information from different sources.

Rijk ðtÞ Z exp Kltijk tb for tR 0: (8)
The transformed failure rate of the system is given as:
p X
X q X
m b0i
lts Z ltijk
iZ1 jZ1 kZ1
The corresponding characteristic life of the system is
expressed as [18]:
( )Kð1=bÞ
Xp X q X m
1
qs Z b
(10)
iZ1 jZ1 kZ1 qijk
The system hazard rate at any time t can be given as
! !
 bK1 
tbK1 tbK1 t qualitative and quantitative information.
lðtÞ Z b b C. Cb b Zb (11)
q111 qijk qbs
This model assumes that the known value of shape factor b
is the same for all interfaces, components, functions, and
failure mechanisms. The model is applicable for different
known values of b with some minor changes in the equations.
5.3. Updating reliability parameters
One of the objectives of this paper is to develop a cost
effective test plan for reliability demonstration purpose, which
means for each identified category of test determine the sample
size requirement to demonstrate the given reliability target. As
we all know, the product design and development is an
evolutionary process, and a vast amount of knowledge and
information is generated during this evolution process. There-
fore, we feel that to make reliability testing a cost effective
process, one must be able to collect and utilize all direct and
indirect pieces of evidence such as warranty data of similar
systems, expert opinions or engineering judgment to assess the
impact of design changes or corrective actions, and incorporate
them into reliability analysis to give current estimates of
reliability parameters. The Bayes’ theorem provides a
mathematical framework for handling prior information as
well as new evidence and continuous updating throughout the
product development process as discussed by authors and
others [12,19].
The reparameterized version of the Weibull distribution, the
assumption of the known b value, allows us to take gamma
distribution as a prior distribution [19], and continuous
The average transformed failure rate based on the prior
distribution is given as:
a0i
lt0i Z (13)
(9)
The new parameters of the posterior distribution are updated
as:
a1j Z a0j C rj and b1j Z b0j C Tj
where rj represents number of failures of jth component or
competing risk and Tj is total test duration (transformed). Ref.
[12] gives detail discussion on fuzzy based reliability
improvement assessment model to update reliability par-
ameters in product development process based on both
5.4. Test plan to demonstrate reliability goal
The number of samples required for a reliability demon-
stration testing at any stage of the product development process
depends on the reliability target. Sample size and duration of
test can be calculated based on the Binomial or Weibull
distributions. Binomial distribution is appropriate for pass or
fail type situations and the Weibull distribution is suitable for
durability type testing to failure strategy. However, the
assumption of known value of shape factor helps us
proactively, and allows us to model the underlying test plan
formulas based on the binomial distribution to determine the
test requirements before the test. A test plan gives the required
number of units to be tested and stopping rule. The stopping
rule defines the amount of time to be accumulated on each unit
or the number of failures. The measure of confidence may be
built into statistically designed test plans, ensuring that the
failure mode in question has not been fixed or reliability
requirements has not been achieved, there is low probability
that test will be passed.
Since reliability at any time is a function of characteristic
life,q, the first step in determining the sample size or test
duration is to convert the given reliability targets into
equivalent characteristic life requirements as given below:
b
R
ðtÞ Z eKðt=qÞ
t
q
Z (14)
ðKln R
ðtÞÞ1=b
 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893 889

Once the equivalent characteristic life has been calculated, characteristic life (q):
the next step is to determine the sample size and test duration  1=b
for reliability demonstration at given confidence level. This 1
k Z K lnð1KCÞ (17)
decision depends on the availability of resources and test plan n
management is willing to afford. There are two most frequently
Form Eq. (17), one can easily calculate the test duration for
used test plans in automotive industry:
equivalent characteristic life q* given by Eq. (14) as follows:
1. Zero-failure test plans and t
kZ
2. Zero or one failure test plans q

t Z kq
(18)
5.4.1. Zero-failure test plans
Eq. (18) gives the test duration required for each unit in a
To develop zero-failure test plans one has to make decision
sample of size n to demonstrate the reliability target at given
on how many units to test and what should be the duration of
confidence level.
test. It is very difficult to answer these two questions
Similarly, if the test duration is known, one can easily
simultaneously. However, if we can assume the answer of
calculate the required sample size using Eqs. (15) and (16) as
one question, the answer to second question can be derived
given below:
easily in order to demonstrate the reliability target at given
confidence level. lnð1KCÞ
Assume that company has n units available for testing. Once n ZK
b (19)
t
q

we know the sample size, the test duration can be calculated as
[5]:
5.4.2. Zero or one failure test plans
ð1KCÞ Z Rn (15) If the management is interested in zero or one failure test
plans, the binomial distribution can be used to calculate

 required sample size or test duration [5].
t b t
R Z exp K Z expðKðkÞb Þ where k Z (16)

q q ð1KCÞ Z Rn C n RnK1 ð1KRÞ (20)

If a log transformation is applied to both sides of Eq. (15), The above equation can be used to calculate the test duration
and terms are rearranged from Eqs. (15)–(16), we obtain the for a given sample size or sample size if test duration is given to
following formula to calculate the ratio of test duration (t) and you in similar way as we have shown for zero-failure test plan.

Steering gear Assembly (System)

Pinion
Valve Bellows and
Housing and Rack Functional
Assembly Tie-Rod
Rack Tube Assembly Reliability
Assemblies
Assembly

Transmits steering torque to rack and

converts it into axial force 0.990567414 0.990567414 0.981223802

Maintains pressurized fluid flow and

0.992935585 0.992935585 0.985921075
Functions

assists steering effort

Prevents contamination and water

0.99056741 0.990567414 0.981223802
ingression

Turns steering wheels and tires &

supports tie rod load and links rack to 0.990567414 0.990567414 0.990567414 0.971968324
outer tie rod

Component Reliability 0.99056741 0.97429203 0.97429203 0.981223802 0.922636032

Fig. 6. Two-dimensional reliability matrix.

890 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893

Microsoft Excel provides excellent tools—solver and goal According to the inspection and test standards provided by
seek—to calculate sample size for zero or one failure test plan. the customer and failure analysis, all reliability and durability
tests can be grouped into four categories to analyze four
6. A case example prominent failure mechanisms or damage behavior, i.e.,

To demonstrate the proposed methodology, we recall our
hydraulic power rack and pinion steering system example
discussed in Ref. [12]. The example of the product
development process for steering system represents a real-life
scenario, but actual numbers have been modified due to
proprietary nature of the data and to demonstrate the
applicability of the methodology.
The intent of the bench test during each phase of the vehicle
program (product development process) is to demonstrate
reliability of the steering gear prior to moving to the next phase.
Reliability and durability tests are created to test endurance and
strength of components/system on vehicle with several
different build combinations and environmental and loading
conditions. The whole idea of these endurance and strength
tests is to analyze the cumulative damage behavior of the
system under different loading and operating conditions, their
impact on system functions, and estimate reliability of the
steering gear system.
Fatigue tests,
Wear tests,
Strength tests, and
Corrosion test
To demonstrate the applicability of the methodology, we
have considered four sub-systems and four sub-functions of the
steering system as a result of physical and functional
decomposition. Fig. 6 represents the two-dimensional mapping
matrix of these sub-systems and functions. Fig. 7 shows three-
dimensional matrix, which maps all three elements/dimensions
of product structure and failure analysis, (i.e. physical,
functional, and time or damage behavior (failure mechan-
isms)).
Generally, the customer specifies the reliability require-
ments and expects suppliers to demonstrate these reliability
requirements according to the specifications mentioned in the
inspection and test standards. In the present case we assume

Failure Mechanisms/Modes
Fatigue Strength Corrosion
Functions Sub-systems Wear failure Reliability
failure failure failure
Pinion Housing and
Rack Tube Assembly
Transmits steering torque
to rack and converts it into Rack Assembly 3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
axial force Valve Assembly 3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
Bellows and Tie-Rod
Assemblies
Pinion Housing and
Rack Tube Assembly
Maintains pressurized fluid
flow and assists steering Rack Assembly 3.221E-10 2.40163E-10 1.4669E-10 0.992935585
effort Valve Assembly 3.221E-10 2.40163E-10 1.4669E-10 0.992935585
Bellows and Tie-Rod
Assemblies
Pinion Housing and
3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
Rack Tube Assembly
Prevents contamination Rack Assembly
and water ingression
Valve Assembly
Bellows and Tie-Rod
3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
Assemblies
Pinion Housing and
Rack Tube Assembly
Turns steering wheels and
tires & Supports tie rod Rack Assembly 3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
load and links rack to outer
Valve Assembly 3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
tie rod
Bellows and Tie-Rod
3.221E-10 2.40163E-10 2.3879E-10 1.4669E-10 0.990567414
Assemblies
Failure mechanism/mode reliability 0.9714274 0.978617286 0.98342385 0.98688477 0.922636032

Fig. 7. Three-dimensional reliability matrix.

 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893 891

Table 1 allocation. Once reliability allocation is completed for the

Criticality index and prior information parameters three-dimensional matrix, transferring appropriate values from
System/fail- Criticality Prior infor- Prior parameters
this matrix can generate the two-dimensional matrix, which
ure mechan- index (u) mation (lt) gives component and functional reliability estimates as shown
isms in Figs. 6 and 7.
a b The allocated reliability estimates are further updated by
System – 9.88!10 K09
3085 389574711 incorporating the available evidence of improvement at any
Fatigue 0.4 3.95!10K09 2.3 633022640 level of the system. The qualitative or fuzzy information,
Wear 0.25 2047!10K09 2.72 1197766380 attributed to design changes and modifications, is processed
Strength 0.2 1.98!10K09 3.37 1853587392
and quantified using fuzzy logic based reliability model to
Corrosion 0.15 1048!10K09 3.45 2536186853
estimate reliability improvement. The quantified qualitative
evidence and additional test results are given in Table 2. Figs. 6
that the customer requirement is that the system must meet a
and 7 illustrate the revised reliability estimates at design
minimum reliability and confidence level of R95/C90 verification stages (DV1) after incorporating all available
measured at the equivalent of 10,000 customer miles (mission evidence, both quantitative and qualitative. Fig. 8 illustrate the
time). The equal apportionment of system reliability require- reliability growth curve from the prior model to DV1 phase.
ment into sub-system, function, and failure mode reliabilities
sets reliability targets of RiZRjZRkZ.987, respectively. 6.1. Test plan for further reliability demonstration
Since, the current design is an evolution of the existing
design resulting from continuous design changes and modifi- To further understand how this framework helps identify
cations, the warranty data of existing design were used as prior critical or weak spots in the design and in planning further
information and the Weibull distribution parameters, (i.e. reliability demonstration testing and/or corrective actions, let
shape factor (b) and characteristic life (q)), were estimated as us consider both matrices shown in Figs. 6 and 7. For example,
follows: the estimated reliability of the pinion housing and rack tube
assembly (see the two-dimensional matrix in Fig. 6) is .9905,
Shape factor (b)Z1.4 which is higher than the given reliability target. The major
Characteristic life (q)Z522,525 mile. function of the pinion housing and rack tube assembly is to
protect other components from contamination or corrosion. But
Further, the analysis of the warranty data gives the
the reliability estimate for this function is still lower than the
frequency of failures attributed to different failure causes.
reliability goal. The other sub-assembly that supports the same
These failure causes are then grouped into four broad
function is bellows and tie-rod, and its reliability estimate is
categories of failure modes or mechanisms such as fatigue,
also slightly lower than the target reliability. The failure
wear, strength, and corrosion. Fig. 5 shows the relative mechanism, which propagates the failure of this function, is
frequency or failure mode distribution of warranty data. corrosion. The reliability estimate of corrosion failure is also
The relative frequency of failures attributed to each failure slightly lower than the given reliability target (see Fig. 7). This
mechanism is treated as criticality index (wk) to allocate the three dimensional investigation and understanding explains
prior system reliability to different failure mechanisms or that reliability assessment of pinion housing and rack tube
competing risks (Rk). Table 1 shows the reliability allocation assembly is meeting the given reliability requirements.
and prior distribution parameters for each failure mechanism. However, the reliability assessment for corrosion failure
The reliability of each failure mechanism or competing risk mechanism is lower and needs further testing to demonstrate
is further allocated to each cell of the three-dimensional matrix reliability target. Therefore, the only candidate for further
in equal proportion to keep the analysis simple. We could have testing is the bellows and tie-rod assembly to demonstrate
differentiated reliability allocation to each cell by assigning the reliability of the identified function and failure mechanism. It
criticality index to it. The criticality index for each cell can be necessitates that we need to develop a reliability demonstration
calculated by using critical number (or RPN) from the FMECA test plan to further test bellows and tie-rod assembly for
document for each interface of function, sub-system and failure corrosion/environmental degradation failure mechanism. Like-
mechanism. For more accurate reliability decomposition we wise, one can easily decide about other test categories and sub-
would encourage reliability engineers to use FMECA systems subjected to those tests. Table 3 shows the reliability
information to calculate criticality index and use it in reliability demonstration test plans for each failure mechanisms and
Table 2
Qualitative and quantitative evidence up to DVI stage

Quantified qualititative Test types (quanti-

information tative information)
Reliability improvement Transformed fail- Transformed mean time Fatigue tests Wear tests (units) Strength tests Corrosion tests
index (RII) ure rate (lt) to failure (MTTF) (units) (units) (units)
30% 6.91EK09 1.45EC08 16 6 16 8
892 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893

0.925 Table 4
Reliabiliy demonstration test plan for given test duration
0.92
Test types Subassemblies Sample size (n) Test duration (t)
Reliability growth

0.915 to test

0.91 Fatigue test Rack assembly 19 50,000

Value assembly
0.905 Bellows and tie-
rod assembly
0.9 Wear test Rack assembly 19 50,000
0.895 reliability Value assembly
Bellows and tie-
0.89 rod assembly
Prior Concept DV Environmental Rack assembly 19 50,000
Product development stages (corrosion) test Value assembly
Bellows and tie-
Fig. 8. Reliability growth curve rod assembly
Strength Rack assembly 19 50,000
(durability) test Value assembly
physical elements subjected to each test category when sample Bellows and tie-
size is given as 10 units. Table 4 gives the sample size for each rod assembly
test category when test duration is assumed to be 50,000 mile
per unit given.
The use of Bayesian model in this methodology provides demonstration test plan. Based on our updated reliability
another opportunity to reduce testing time and cost. The use of estimate, if we decide to further demonstrate the reliability
fuzzy logic based Bayesian model—allowing for full use of all target at 50% confidence only, we need maximum four samples
prior information on previous test data, qualitative information to demonstrate the given reliability target which turns out to be
resulting from design changes and corrective actions, and on big saving in both time and cost.
field performance data on similar products—allows for reduced Further, the failure to demonstrate the reliability targets
sample size by one unit. If we include this consideration in our should trigger the further analysis to identify the causes of
test plan, it results in saving four sample and more than 200, design failure and accordingly decide appropriate design
000 mile of test duration. Another way to get advantage of changes or corrective actions. This process continues until
Bayesian updates is that it builds our confidence in product reliability targets are demonstrated for all physical elements
reliability with every new update and, therefore, we can relax and failure mechanisms.
confidence requirement for further reliability demonstration.
For example, the updated reliability estimate for corrosion
failure mechanism is 0.9868, which is very close to the given 7. Conclusions
reliability target. Therefore, this estimate enhances our
confidence, to certain extent, in product performance against The three dimensional understanding and analysis gives an
corrosion failure mode and hence, we can reduce confidence explicit and separate consideration for each single point failure
level requirement for the formulation of reliability and also helps identify the weak links in the design for further
actions in order to improve reliability. The framework helps
Table 3 identify what types of test to be performed and which physical
Reliability demonstration test plan for given sample size element is subjected to the identified test based on the
Test types Subassemblies Sample size (n) Test duration (t)
knowledge of potential failure mechanism and failure analysis
results. The allocation of system level reliability requirements
Fatigue test Rack assembly 10 77,500
Value assembly
to each element forces the design engineers to understand and
Bellows and tie- develop the relationship between component, function, failure
rod assembly mode, and system reliability. It also forces the design engineer
Wear test Rack assembly 10 77,500 to consider reliability equally with other system parameters
Value assembly
Bellows and tie-
such as weight, cost, and other performance characteristics.
rod assembly The methodology provides effective link between failure
Environmental Rack assembly 10 77,500 analysis process (such as FMECA) and reliability demon-
(corrosion) test Value assembly stration plan through criticality index and ensures focus on
Bellows and tie-
failure modes and their elimination throughout the product
rod assembly
Strength Rack assembly 10 77,500 development process. Finally, the consideration of prior
(durability) test Value assembly information in reliability updates and subsequent reliability
Bellows and tie- demonstration test plan helps in focusing on critical elements
rod assembly and saves resources and time.
 O.P. Yadav et al. / Reliability Engineering and System Safety 91 (2006) 882–893
Acknowledgements
The authors would like to express their sincere appreciation
for the many constructive and useful comments and sugges-
tions offered by the reviewers.
References
[1] O’Conner P. Test for reliability. Qual Reliab Eng Int 2003;19:73–84.
[2] Usher JS, Alexander SM, Thompson JD. System reliability prediction
based on historical data. Qual Reliab Eng Int 1990;6:209–18.
[3] Wang CJ. Sample size determination of bogey tests without failure. Qual
Reliab Eng Int 1991;7:35–8.
[4] Wang CJ, Lu M-W. A two-stage sampling plan for bogey tests. Qual
Reliab Eng Int 1992;8:29–35.
[5] Abernethy RB. In: Abernethy Robert A, editor. The new weibull
handbook. 3rd ed; 1998.
[6] Goel PS, Singh N, Kumar P. A mathematical model for design and
production verification planning. SAE technical paper series. Paper offer
no. 992AM-10; 1999.
[7] Porter A. Accelerated reliability qualification in automotive testing. Qual
Reliab Eng Int 2004;20:115–20.
[8] Gerling WH, Preussger A, Wulfert FW. Reliability qualification of
semiconductor devices based on physics-of-failure and risk and
opportunity assessment. Qual Reliab Eng Int 2002;18:81–98.
893
[9] Jensen F. Component failures based on flaw distribution. Proc Annu
Reliab Maint Symp 1989.
[10] Mortin DE, Krolewski JG, Cushing MJ. Consideration of component
failure mechanisms in the reliability assessment of electronic equip-
ment—addressing the constant failure rate assumption. Proc Annu Reliab
Maint Symp 1995.
[11] Pecht M, Dasgupta A, Barker D, Leonard CT. The reliability physics
approach to failure prediction modeling. Qual Reliab Eng Int 1990;6:
267–73.
[12] Yadav OP, Singh N, Chinnam RB, Goel PS. A fuzzy logic based approach
to reliability improvement estimation during product development. Reliab
Eng Syst Saf 2003;80:63–74.
[13] Yadav OP, Singh N, Goel PS. A practical approach to system
reliability growth modeling and improvement. Proc Annu Reliab Maint
Symp 2003.
[14] Kapur KC, Lamberson LR. Reliability in engineering design. New York:
Wiley; 1977.
[15] MIL-STD-1629. Standard procedure for performing a failure modes and
effects analysis. http://www.store.mil-standards.com.
[16] Soland RM. Bayesian analysis of the weibull process with unknown scale
parameter and its application to acceptance sampling. IEEE Trans Reliab
1968;R-17:84–90.
[17] Hoyland A, Rausand M. System reliability theory: models and statistical
methods. New York: Wiley; 1994.
[18] Nelson W. Applied life data analysis. New York: Wiley; 1982.
[19] Martz HF, Waller RA. Bayesian reliability analysis. New York: Wiley;
1982.