Probability and Statistics with

Reliability, Queuing and Computer

Science Applications: Chapter 3
Continuous Random Variables

Definitions
Distribution function:
If FX(x) is a continuous function of x, then X is a
continuous random variable.
FX(x): discrete in x Discrete rvs
FX(x): piecewise continuous Mixed rvs

Definitions

(Continued)

Equivalence:
CDF (cumulative distribution function)
PDF (probability distribution function)
Distribution function
FX(x) or FX(t) or F(t)

Probability Density Function (pdf)

X : continuous rv, then,

pdf properties:
1.
2.

Definitions
(Continued)

Equivalence: pdf
probability density function
density function
density
f(t) = dF
dt

F (t )

f ( x)dx

f ( x)dx ,
0

For a non-negative
random variable

Exponential Distribution

Arises commonly in reliability & queuing theory.

A non-negative random variable
It exhibits memoryless (Markov) property.
Related to (the discrete) Poisson distribution
Interarrival time between two IP packets (or voice calls)
Time to failure, time to repair etc.

Mathematically (CDF and pdf, respectively):

CDF of exponentially distributed

random variable with = 0.0001

F(t)

12500

25000

37500

50000

Exponential Density Function

(pdf)

f(t)

Memoryless property
Assume X > t. We have observed that the component has

not failed until time t.

Let Y = X - t , the remaining (residual) lifetime
The distribution of the remaining life, Y, does not depend

on how long the component has been operating.

Distribution of Y is identical to that of X.

Memoryless property
Assume X > t. We have observed that the
component has not failed until time t.
Let Y = X - t , the remaining (residual)
lifetime

Gt ( y ) P (Y y | X t )
P( X y t | X t )
P (t X y t )
y

1 e
P( X t )

Memoryless property (Continued)

Thus Gt(y) is independent of t and is identical to the
original exponential distribution of X.
The distribution of the remaining life does not depend
on how long the component has been operating.
Its eventual breakdown is the result of some suddenly
appearing failure, not of gradual deterioration.

Reliability as a Function of Time

Reliability R(t): failure occurs after time t. Let X
be the lifetime of a component subject to failures.

Let N0: total no. of components (fixed); Ns(t):

surviving ones; Nf(t): failed one by time t.

Definitions

(Continued)

Equivalence:
Reliability
Complementary distribution function
Survivor function
R(t) = 1 -F(t)

Failure Rate or Hazard Rate

Instantaneous failure rate: h(t) (#failures/10k hrs)

Let the rv X be EXP( ). Then,

Using simple calculus the following apples to any rv,

Hazard Rate and the pdf

f (t )
f (t )
h (t )

R (t ) 1 F (t )
h(t) t = Conditional Prob. system will fail in
(t, t + t) given that it has survived until time t
f(t) t = Unconditional Prob. System will fail in
(t, t + t)
Difference between:
probability that someone will die between 90 and 91,
probability that someone will die between 90 and 91

given that he lives to 90

Weibull Distribution
Frequently used to model fatigue failure, ball bearing failure etc.
(very long tails)

t
Reliability:
R t e
t0
Weibull distribution is capable of modeling DFR ( < 1), CFR ( =
1) and IFR ( >1) behavior.

is called the shape parameter and is the scale parameter

Failure rate of the weibull

distribution with various values
of and = 1

5.0

1.0

2.0

3.0

4.0

Infant Mortality Effects in System

Modeling
Bathtub curves
Early-life period
Steady-state period
Wear out period

Failure rate models

Bathtub Curve

Failure Rate (t)

Until now we assumed that failure rate of equipment is time (age)

independent. In real-life, variation as per the bathtub shape has been
observed

Infant Mortality
(Early Life Failures)

Steady State
Operating Time

Wear out

Early-life Period
Also called infant mortality phase or reliability
growth phase
Caused by undetected hardware/software defects
that are being fixed resulting in reliability growth
Can cause significant prediction errors if steadystate failure rates are used
Availability models can be constructed and solved
to include this effect
Weibull Model can be used

Steady-state Period
Failure rate much lower than in early-life period
Either constant (age independent) or slowly
varying failure rate
Failures caused by environmental shocks
Arrival process of environmental shocks can be
assumed to be a Poisson process
Hence time between two shocks has the
exponential distribution

Wear out Period

Failure rate increases rapidly with age
Properly qualified electronic hardware do
not exhibit wear out failure during its
intended service life (Motorola)
Applicable for mechanical and other
systems
Weibull Failure Model can be used

Bathtub curve
DFR phase: Initial design, constant bug fixes
CFR phase: Normal operational phase
IFR phase: Aging behavior

h(t)

(burn-in-period)

(wear-out-phase)

CFR
(useful life)
DFR

IFR
t
Decreasing failure rate

Increasing fail. rate

Failure Rate Models

We use a truncated Weibull Model

Failure-Rate Multiplier

7
6
5
4
3
2
1
0

2,190

4,380

6,570 8,760 10,950 13,140 15,330 17,520

Operating Times (hrs)

Infant mortality phase modeled by DFR Weibull and the steady-state

phase by the exponential

Failure Rate Models (cont.)

This model has the form:
1 t 8,760
W ( t ) C 1 t
SS
t 8,760
where:
Csteady-state

failure
rate
1 W 1 , SS
is the Weibull shape parameter
Failure rate multiplier = W ( t) SS

Failure Rate Models (cont.)

There are several ways to incorporate time dependent failure rates in

availability models
The easiest way is to approximate a continuous function by a
decreasing step function
Failure-Rate Multiplier

7
6

5
4
3

2
1
0

2,190

4,380

SS

6,570 8,760 10,950 13,140 15,330 17,520

Operating Times (hrs)

Failure Rate Models (cont.)

Here the discrete failure-rate model is
defined by:
0 t 4,380
W ( t ) 1
2
4,380 t 8,760
ss
t 8,760

Uniform Random Variable

All (pseudo) random generators generate random
deviates of U(0,1) distribution; that is, if you
generate a large number of random variables and
plot their empirical distribution function, it will
approach this distribution in the limit.
U(a,b) pdf constant over the (a,b) interval and
CDF is the ramp function

Uniform density
U(0,1) pdf
1.2
1

cdf

0.8
0.6
0.4
0.2
0
0

0.1 0.2 0.3 0.6 0.7 0.8 0.9

1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9

tim e

2.1 2.2

Uniform distribution
The distribution function is given by:
0 ,

F(x)=

xa
b 1a ,

x < a,
a <x<b
x > b.

Uniform distribution (Continued)

HypoExponential
HypoExp: multiple Exp stages in series.
2-stage HypoExp denoted as HYPO(1, 2). The density,
distribution and hazard rate function are:

HypoExp results in IFR: 0 min(1, 2)

Disk service time may be modeled as a 3-stage Hypoexponential
as the overall time is the sum of the seek, the latency and the
transfer time

HypoExponential used in software

rejuvenation models
Preventive maintenance is useful only if failure rate is increasing
A simple and useful model of increasing failure
rate:
Failure
Robust
state

probable
state

Failed state

Time to failure: Hypo-exponential distribution

Increasing failure rate

aging

Erlang Distribution
Special case of HypoExp: All stages have same rate.

[X > t] = [Nt < r] (Nt : no. of stresses applied in (0,t])

and Nt is Possion (param t). This interpretation gives,

Erlang Distribution
Is used to approximate the deterministic one
since if you keep the same mean but increase
the number of stages, the pdf approaches the
delta function in the limit
Can also be used to approximate the uniform
distribution

probability density functions (pdf)

If we vary r keeping r/ constant, pdf of r-stage Erlang approaches

an impulse function at r/ .

cumulative distribution functions (cdf)

And the cdf approaches a step function at r/. In other words

r-stage Erlang can approximate a deterministic variable.

1.8

Comparison of
probability density functions (pdf)

1.6
1.4
1.2
1
pdf

3-stage Erlang pdf

U(0,1) pdf

0.8
0.6
0.4
0.2
0

T=1
time

1.2

Comparison of cumulative distribution

functions (cdf)

cdf

0.8

3-stage Erlang cdf

0.6

U(0,1) cdf

0.4

0.2

T=1
time

Gamma Random Variable

Gamma density function is,

Gamma distribution can capture all three failure modes,

viz. DFR, CFR and IFR.
= 1: CFR
<1 : DFR
>1 : IFR
Gamma with = and = n/2 is known as the chisquare random variable with n degrees of freedom

HyperExponential Distribution
Hypo or Erlang Sequential Exp( ) stages.
Alternate Exp( ) stages HyperExponential.

CPU service time may be modeled as HyperExp

In workload based software rejuvenation model we found the sojourn
times in many workload states have this distribution

Log-logistic Distribution
Log-logistic can model DFR, CFR and IFR failure models
simultaneously, unlike previous ones.

For, > 1, the failure rate first increases with t (IFR); after
momentarily leveling off (CFR), it decreases (DFR) with
time. This is known as the inverse bath tub shape curve
Use in modeling software reliability growth

Hazard rate comparison

Defective Distribution
If
Example:

This defect (also known as the mass at infinity) could

represent the probability that the program will not terminate
(1-c). Continuous part can model completion time of
program.
There can also be a mass at origin.

Pareto Random Variable

Also known as the power law or long-tailed
distribution
Found to be useful in modeling

CPU time consumed by a request

Webfile sizes
Number of data bytes in FTP bursts
Thinking time of a Web browser

Gaussian (Normal) Distribution

Bell shaped pdf intuitively pleasing!
Central Limit Theorem: mean of a large number of
mutually independent rvs (having arbitrary
distributions) starts following Normal distribution
as n
: mean, : std. deviation, 2: variance (N(, 2))
and completely describe the statistics. This is
significant in statistical estimation/signal
processing/communication theory etc.

Normal Distribution (contd.)

N(0,1) is called normalized Guassian.
N(0,1) is symmetric i.e.
f(x)=f(-x)
F(z) = 1-F(z).
Failure rate h(t) follows IFR behavior.
Hence, N( ) is suitable for modeling long-term wear or
aging related failure phenomena.

Functions of Random Variables

Often, rvs need to be transformed/operated upon.
Y = (X) : so, what is the density of Y ?
Example: Y = X2

If X is N(0,1), then,

Above Y is also known as the 2 distribution (with 1-degree of

freedom).

Functions of RVs (contd.)

If X is uniformly distributed, then, Y= --1ln(1-X)
follows Exp() distribution
transformations may be used to generate random
variates (or deviates) with desired distributions.

Functions of RVs (contd.)

Given,
A monotone differentiable function,

Above method suggests a way to get the random variates with desired distribution.
Choose to be F.
Since, Y=F(X), FY(y) = y and Y is U(0,1).
To generate a random variate with X having desired distribution, generate U(0,1)
random variable Y, then transform y to x= F-1(y) .
This inversion can be done in closed-form, graphically or using a table.

Jointly Distributed RVs

Joint Distribution Function:

Independent rvs: iff the following holds:

Joint Distribution Properties

Joint Distribution Properties (contd)

Order statistics: kofn, TMR

Order Statistics: KofN

X1 ,X2 ,..., Xn iid (independent and identically distributed) random
variables with a common distribution function F().
Let Y1 ,Y2 ,...,Yn be random variables obtained by permuting the set
X1 ,X2 ,..., Xn so as to be in
increasing order.
To be specific:
Y1 = min{X1 ,X2 ,..., Xn} and
Yn = max{X1 ,X2 ,..., Xn}

Order Statistics: KofN

(Continued)

The random variable Yk is called the k-th ORDER

STATISTIC.
If Xi is the lifetime of the i-th component in a system of n
components. Then:
Y1 will be the overall series system lifetime.
Yn will denote the lifetime of a parallel system.
Yn-k+1 will be the lifetime of an k-out-of-n system.

Order Statistics: KofN (Continued)

To derive the distribution function of Yk, we note that the
probability that exactly j of the Xi's lie in (- ,y] and (n-j)
lie in (y, ) is:

n j

F ( y ) [1 F ( y )]
j

hence
j

j k

FYk ( y )

n j

F ( y ) [1 F ( y )]
j

Applications of order statistics

Reliability of a k out of n system

Rkofn (t ) ( nj )[ R (t )] j [1 R (t )]n j
j k

Series system:

Rseries (t ) [ R (t )]

or Ri (t )
i 1

Parallel system:
n
R
(
t
)

[
1

R
(
t
)]
Minimum of n EXP random
variables is special case of Y1 = min{X1,
parallel
,Xn} where Xi~EXP(i)
Y1~EXP( i)
This is not true (that is EXP dist.) for the parallel case

Triple Modular Redundancy (TMR)

R(t)
R(t)

Voter

R(t)

An interesting case of order statistics occurs when

we consider the Triple Modular Redundant (TMR)
system (n = 3 and k = 2). Y2 then denotes the time
until the second component fails. We get:

RTMR (t ) 3R (t ) 2 R (t )
2

TMR (Continued)
Assuming that the reliability of a single
component is given by,

we get:

RTMR (t ) 3e

2 t

2e

3t

TMR

(Continued)

In the following figure, we have plotted

RTMR(t) vs t as well as R(t) vs t.

TMR

(Continued)

Cold standby (dynamic redundancy)

X

Lifetime of Lifetime of
Spare
Active
EXP()
EXP()
Total lifetime 2-Stage Erlang

R(t ) (1 t )e

EXP()

Assumptions: Detection & Switching perfect; spare

does not fail

EXP()

Sum of RVs: Standby Redundancy

Two independent components, X and Y
Series system (Z=min(X,Y))
Parallel System (Z=max(X,Y))
Cold standby: the life time Z=X+Y

Sum of Random Variables

Z = (X, Y) ((X, Y) may not be independent)

For the special case, Z = X + Y

The resulting pdf is (assuming independence),

Convolution integral (modify for the non-negative case)

Convolution (non-negative case)

Z = X + Y, X & Y are independent random
variables (in this case, non-negative)
t

f Z (t ) f X ( x) fY (t x) dx
0

The above integral is often called the

convolution of fX and fY. Thus the density of the
sum of two non-negative independent,
continuous random variables is the convolution
of the individual densities.

Cold standby derivation

X and Y are both EXP() and independent.
Then
t

f t (t ) e e
x

( t x )

dx

2 t

dx
0

te , t 0
2

Cold standby derivation

(Continued)

Z is two-stage Erlang Distributed

t

FZ (t ) f Z ( z )dz 1 (1 t )e
0

R(t ) 1 F (t )
t

(1 t )e , t 0

Convolution: Erlang
Distribution
The general case of r-stage Erlang Distribution
When r sequential phases have independent identical exponential
distributions, then the resulting density is known as r-stage (or rphase) Erlang and is given by:

Convolution: Erlang
EXP()

EXP()

(Continued)

EXP()

r r 1 t

t e
f (t )
(r 1)!

( t ) k t
F (t ) 1
e
k!
k 0
r 1

Warm standby
With Warm spare, we have:
Active unit time-to-failure: EXP()
Spare unit time-to-failure: EXP()

EXP(+ )

EXP()

2-stage hypoexponential distribution

Warm standby derivation

First event to occur is that either the active or the spare will fail. Time
to this event is min{EXP(),EXP()} which is EXP( + ).
Then due to the memoryless property of the exponential, remaining
time is still EXP().
Hence system lifetime has a two-stage hypoexponential distribution
with parameters

1 = + and 2 = .

Warm standby derivation

(Continued)

X is EXP(1) and Y is EXP(2) and are

independent
1 = 2
Then fZ(t) is
t

f Z (t ) 1e

1 x

2 e

2 ( t x )

dx

12 2t 12 1t

e
e
1 2
2 1

Hot standby
With hot spare, we have:
Active unit time-to-failure: EXP()
Spare unit time-to-failure: EXP()

EXP(2)

EXP()

2-stage hypoexponential

TMR and TMR/simplex

as hypoexponentials
TMR/Simplex
EXP(3)

EXP()

TMR
EXP(3)

EXP(2)

Hypoexponential: general case

r

are mutually independent and X is exponentially

Z=

i 1

, where X1 ,X2 , , Xr
i

distributed with parameter i

(i = j for i = j).

Then Z is a r-stage hypoexponentially

distributed random variable.

EXP(1)

EXP(2)

EXP(r)

Hypoexponential: general case

KofN system lifetime as a

hypoexponential
At least, k out of n units should be operational for the
system to be Up.
EXP(n)

Y1

EXP((n-1))

Y2

...

EXP(k)

Yn-k+1

EXP((k-1))

Yn-k+2

...

EXP()

Yn

KofN with warm spares

At least, k out of n + s units should be operational
for the system to be Up. Initially n units are active
and s units are warm spares.
EXP(n
s)

EXP(n
+(s-1) )

...

EXP(n
+ )

EXP(n)

...

EXP(k)

Sum of Normal Random Variables

X1, X2, .., Xk are normal iid rvs, then, the rv
Z = (X1+ X2+ ..+Xk) is also normal with,

X1, X2, .., Xk are normal. Then,

follows Gamma
or the 2 (with n-degrees
of freedom) distribution