2.

PRELIMINARIES
Laplace Transforms, Moment Generating
Functions and Characteristic Functions
2.1 Definitions
2.2 Theorems on Laplace Transforms
2.3 Operations on Laplace Transforms
2.4 Limit Theorems
2.5 Dirac Delta Function
2.6 Appendix: Complex Numbers
2.7 Appendix: Notes on Partial Fractions

31

Laplace Transforms, Moment Generating

Functions and Characteristic Functions
2.1. Definitions:
Let (t) be defined on real line.
a. Moment Generating Function (MGF)
Z
M GF =
et (t)dt

which exists if

|et (t)dt| <

32

b. Characteristic Function (CF)

Z
CF =
eiyt (t)dt

which exists if

|eiyt (t)|dt <

Since eiyt = cos yt + i sin yt,

Z
|(t)|dt <
CF exists if

|eiyt | = 1

33

(i2 = 1).

c. Laplace Transform
Let (t) be defined on [0, ) and assume
Z
ex0 t (t)dt <
0

for some value of x0 0.

Characteristic function of ex0 t (t) is
Z
eiyt ex0 t (t)dt (sign of iyt could be + or -).
CF =
0
Z
e(x0 +iy)t (t)dt
CF =
0

CF

x0 + iy

(s) =

est (t)dt for R(s) x0

(s) is Laplace Transform of (t).

34

 (s) =

est (t)dt
0

Since (t) defined on [0, ), for 0

Z
Z
est (t)dt =
(s) = lim
0

est (t)dt.
0+

Often a script L is used to denote a LaPlace transform; i.e.

L {(t)} = (s)
Suppose f (t) is a pdf on [0, ),
Z
f (s) =
est f (t)dt , R(s) 0
0

Laplace transform is appropriate for non-negative r.v.

35

If the cdf is F (t) =

f (x)dx =
0

f (s) =

Z
Z

dF (x)
0

est dF (t)
0

is Laplace-Stieltjes Transform.

Note: E(esT ) = f (s)

f (0) = 1, 0 f (s) 1

36

2.2 Theorems on Laplace Transforms (LT)

a. Uniqueness Theorem.
Distinct probability distributions have distinct Laplace Transforms
b. Continuity Theorem
For n = 1, 2, . . . , let {Fn (t)} be a sequence of cdf 0 s such that Fn F .

Define {fn (s)} as the

sequence
of
LT
such
that
L
{f
(t)}
=
f
(s)
n
n
Z

est dF (t).

and define f (s) =

Then fn (s) f (s) and conversely if fn (s) f (s), then

Fn (t) F (t)

37

c. Convolution Theorem
If T1 , T2 are independent, non-negative r.v. with p.d.f f1 (t), f2 (t) then
the pdf of T = T1 + T2 is
Z
f1 ( )f2 (t )d
f (t) =
0

and L {f (t)} = f1 (s)f2 (s)

In general if {Ti } i = 1, 2, . . . , n are independent non-negative r.v.,
then the Laplace transform of the pdf of T = T1 + + Tn is
n
Y
fi (s)
L {f (t)} =
i=1

38

d. Moment Generating Property

Suppose all moments exist.

f (s) =

st

f (t)dt =

=
=

n=0

X
n=0

(1)

n=0

n
s
(1)n
n!

s n tn
n!

f (t)dt

tn f (t)dt
0

sn
mn where mn = E(T n )
(1)
n!

39

e. Inversion Theorem
Knowledge of f (s). The inversion formula is written
Z c+i
1
est f (s)ds
f (t) =
2i ci
where the integration is in the complex plane and c is an appropriate
constant. (It is beyond the scope of this course to discuss the inversion
formula in detail).
Notation: L {f (t)} = f (s)
f (t) = L1 {f (s)}
where L1 { } is referred to as the Inverse Laplace Transform.

40

Example: Exponential Distribution

f (t)

E(T )

f (s)

et for t 0

1/, V (t) = 1/2 .

Z
Z
est et dt =
0

=
f (s)

+s

(1)n

e(s+)t dt
0

( + s)e

(+s)t

=
+s
1+

n=0

Z

s
n!

n=0

n!
n

mn = E(T n ) =

41

n!
n

(1)n

dt =
+s

 s n

L{e

,
}=
+s

} = et
{
+s

L{et } = ( + s)1 , L1 {( + s)1 } = et

Note:
Z c+i
1
est f (s)ds = f (t)
2i ci


Z c+i

1
st
ds = et
e
2i ci
+s
Z c+i
1
est ( + s)1 ds = et
2i ci
L1 {( + s)1 }

42

= et

L1 {( + s)1 } = et
Differentiating w.r. to
L1 {( + s)2 } = tet
again

L1 {2( + s)3 } = t2 et

..
.
(n 1) times

and

L1 {(n 1)!( + s)n } = tn1 et

L

tn1 et
(n)

= ( + s)n , (n) = (n 1)!

n1 t

e
(t)
(n)

43

= n /( + s)n

(t)n1 et
is gamma distribution.
Note:
(n)

If {Ti } i = 1, 2, . . . , n are independent non-negative identically

distributed r.v. following an exponential distribution with parameter and
T = T1 + . . . + T n

n
n
Y

f (s) =
fi (s) =
+s
1

44

Example: Suppose T1 , T2 are independent non-negative random variables

following exponential distributions with parameters 1 , 2 (1 6= 2 ).
What is distribution of T = T1 + T2 ?



1
2
f (s) = f1 (s)f2 (s) =
.
1 + s
2 + s
f (s) can be written as a partial fraction; i.e.

f (s)
where

A1

A2
A1 (2 + s) + A2 (1 + s)
A1
+
=
(1 + s) (2 + s)
(1 + s)(2 + s)

1 2
, A2 = A1
2 1

45

Since L1 {( + s)1 } = et
f (t) =

L1 {f (s)} =
=
=

A1 L1 {(1 + s)1 } + A2 L1 {(2 + s)1 }

A1 e1 t + A2 e2 t

1 2  1 t
2 t
e
e
2 1

Suppose T1 , T2 , T3 are non-negative independent r.v. so that T1 is

exponential with parameter 1 and T2 , T3 are exponential each with
parameter 2 . Find pdf of T = T1 + T2 + T3 .

2

2
B1
B2
A1
1

+
+
=
f (s) =
1 + s
2 + s
1 + s 2 + s (2 + s)2
f (t) = A1 e1 t + B1 e2 t + B2 te2 t

46

Homework:
Show A1 = (1 2 )2 1 22


1
2
B1 = (1 2 ) + (1 2 )
1 22
B2 = (1 2 )1 1 22

In general if fi (s) = (i /i + s) i = 1, 2, . . . , n and

Qn

f (s) = i=1 fi (s), then f(t) is called the Erlangian distribution.

(Erlang was a Danish telephone engineer who used this distribution to
model telephone calls).

47

2.3 Operations on Laplace Transforms

L{et }

L{tn1 et }

( + s)1

(n)/( + s)n

Setting = 0 in above

L{1}

= 1/s

L{tn1 }

= (n)/sn

48

Homework: Prove the following relationships


Z t
(s)
(x)dx =
1. L
s
0
2. L {0 (t)} = s (s) (0+ )
3. L {(r) (t)} = sr (s) sr1 (0+ ) sr2 0 (0+ )
. . . (0) (0+ )
Prove (1) and (2) using integration by parts
Prove (3) by successive use of (2).

49

Suppose f (t) is a pdf for a non-negative random variable and

Z
Z t
f (x)dx, Q(t) =
f (x)dx
F (t) =
0

Since L {

Rt
0

(s)
(x)dx} = s

f
(s)
L {F (t)} = F (s) = s

Since Q(t) = 1 F (t)

f
(s)
1
L {Q(t)} = Q (s) = s F (s) =
s

50

2.4 Limit Theorems

lim (t) = lim s (s)

t0+

Proof:
L {0 (t)} = s (s) (0+ )
0

lim (s) = lim [s (s) (0+ )]

But lim

est 0 (t)dt = 0
0

.. lim [s (s) (0+ )] = 0

s

51

lim (t) = lim s (s)

s0

Proof:
0

lim L { (t)} = lim

s0

s0

= lim

st

(t)dt =

0 (t)dt
0

t
0

0 (x)dx = lim [(t) (0+ )].

t

Since L {0 (t)} = s (s) (0+ )

lim [s (s) (0+ )] = lim [(t) (0+ )]
t

s0

lim s (s) = lim (t)

t

s0

52

2.5. Dirac Delta Function

Sometimes it is useful to use the Dirac delta function. It is a strange
function and has the property.

0 for t 6= 0
(t) =
t=0
Define

Let h > 0

1 if t > 0
U (t) =
0 t0

U (t + h) U (t)
1
=
(t, h) =

h
h

0
53

if t > 0
if h < t 0
if t h

Define

for t 6= 0
U (t + h) U (t) 0
=
(t) = lim (t, h) = lim
for t = 0
h0
h0
h

Consider
Z
0

(x)(t x)dx

= lim

h0


U (t x + h) U (t x)
(x) lim
dx
h0
h
1
h

Z

(x)U (t x + h)dx

1
Since U (t x + h) =
0

tx+h>0
otherwise
54

(x)U (t x)dx

(x)(t x)dx
0
)
(Z
Z t
t+h
1
(x)dx
(x)dx
= lim
h0 h
0
0
= lim

h0

1
h

(Z

t+h

(x)dx
t

= lim

h0

(t + h)h
h

= (t)

(by mean value theorem)

for some value of 0 < < 1.
Z
(x)(t x)dx = (t)
..
0

Similarly (t) =

(t x)(x)dx
55

(t) =

(x)(t x)dx =

(t x)(x)dx

Special Cases
(x) = 1 for all x
1=

(t x)dx =

(x)dx for all t

0

(x)dx = 1
0

Suppose (x) = esx . Then

Z
If t = 0,

esx (x t)dx = est

esx (x)dx = L{(x)} = 1

56

Example 1 Consider a non-negative random variable T such that

pn = P {T = tn }. Define

f (t) =

n=1

f (t)dt =
0

pn

n=1

F (tn ) = P {T tn } =

X
j=1

pn (t tn )

(t tn )dt =

pj U (tn+1 tj ) =

pn = 1

tn

f (t)dt =
0

pj

jn

Note: If tj < tn+1 tj tn

f (s) =

est f (t)dt =
0

n=1

pn

57

est (t tn )dt =

n=1

pn estn

Example 2
Consider a random variable T such that p = P {T = 0} but for T > 0
Z t2
q(x)dx
P {t1 < T t2 } =
t1

f (t)
f (s)

= p(t) + (1 p)q(t)
= p + (1 p)q (s).

Another way of formulating this problem is to define the random variable

T by

0 with probability p
T =
t with pdf q(t) for t > 0

This is an example of a random variable having both a discrete and

continuous part.
58

2.6 Appendix
ELEMENTS OF COMPLEX NUMBERS

r(x, y)

r
r

>

>

Rectangular

Polar Coordinates

Coordinates

r = Modulus
= Amplitude

x = r cos ,

y = r sin

59

Complex no. representation:

p
z = x + iy, r = x2 + y 2
= tan1

|z| = absolute value= r

x
y

i: to be determined

z = r(cos + i sin ) = rei

Suppose zj = aj + ibj
Addition/Subtraction: z1 z2 (a1 a2 ) + i(b1 b2 )

60

Multiplication:
zj = rj eij , z1 z2 = r1 r2 ei(1 +2 ) z1 z2 r = r1 r2 ,

= 1 + 2

z1 z2 = r1 r2 (cos(1 + 2 ) + i sin(1 + 2 ) = x + iy
x = r1 r2 cos(1 + 2 ) = r1 r2 (cos 1 cos 2 sin 1 sin 2 ) = a1 a2 b1 b2
y = r1 r2 sin(1 + 2 ) = r1 r2 (sin 1 cos 2 + cos 1 sin 2 ) = b1 a2 + a1 b2
z1 z2 = (a1 + ib1 )(a2 + ib2 ) = a1 a2 + i[a1 b2 + a2 b1 ] + i2 [b1 b2 ]
If i2 = 1 Re(z1 z2 ) = a1 a2 b1 b2
i, i2 = 1,

i4n+1 = i,

i3 = i,

i4 = 1

i4n+2 = 1,

i4n+3 = i,

i4n = 1

z n = [r(cos + i sin )]n = rn (cos n + i sin n)

61

Exponential function
e

X
n
0

ei

ei
ei

n!

i3 3
i2 2
+
+ ...
= 1 + i +
2
3!
4
6
3
5
2
+

+ . . . ) + i(
+
...)
= (1
2
4!
6!
3!
5!
= cos + i sin ,
= cos i sin , cos() = cos ; sin = sin
ei ei
ei + ei
, sin =
cos =
2
2i
cos 0 = 1, sin 0 = 0

Since ei = cos + i sin ; z = rei

If z = ex+iy = ex eiy :

ex : modulus, ey : amplitude

z1 z2 = ex1 +iy1 ex2 +iy2 = ex1 +x2 +i(y1 +y2 )

62

MOMENT GENERATING AND

CHARACTERISTIC FUNCTIONS

tY

MGF: (t) = E(e ) = ety dF (y)

R
(t) exists if | ety | dF (y)

Sometimes mgf does not exist.

Importance of M GF
Uniqueness Theorem: If two random variables have the same mgf , they
have the same cdf except possibly at a countable number of points having
0 probability.

63

Continuity Theorem: Let {Xn } and X have mgf {n (t)} and (t) with
cdf 0 s Fn (x) and F (x). Then a necessary and sufficient condition for
lim Fn (X) = F (X) is that for every t, limn n (t) = (t), where
n

(t) is continuous at t = 0.
Inversion Formula: Knowledge of (t) enables the pdf or frequency
function to be calculated.
Convolution Theorem: If Yi are independent with mgf i (t), then the
n
n
X
Y
Yi is S (t) =
i (t)
mgf of S =
1

If i (t) = (t)

S (t) = (t)n

64

MOMENT GENERATING FUNCTIONS

DO NOT ALWAYS EXIST!

For that reason one ordinarily uses a characteristic function of a

distribution rather than the mgf .

function of a random variable y is

Def. The characteristic
Z

(t) = E(eity ) =



ity 

|(t)| = E(e )
 ity 
as e  = 1.

eity dF (y) for < t <

 ity 
e  dF (y) =

Characteristic Functions always exist.

65

dF (y) = 1

Relation between mgf and cf .

(t) = (it)

(t) =

X (it)n
n!

(r) (0) = ir mr

mn

The characteristic function is a Fourier Transform; i.e. for any function

g(y)
F.T.(q(y)) =

eity q(y)dy

66

if q(y) is pdf c.f.

RELATION TO LaPLACE TRANSFORMS

Let Y be a non-negative r.v. with pdf f (y); i.e. P {Y 0} = 1. Then it is
common to use Laplace Transforms instead of characteristic functions.
Def.: s = a + ib (a > 0). Then the Laplace Transform of f (y) is
Z
esy f (y)dy
f (s) =
0

Since esy f (y) = eiby eay f (y)

The Laplace Transform is the equivalent of taking the Fourier Transform
of eay f (y)
Ex. Exponential (t) = (1
Z
f (s) =
esy ey dy =
0

it 1
)

= (1 + )1
+s

67

MOMENT GENERATING FUNCTIONS AND

CHARACTERISTIC FUNCTIONS

mgf

cf

Bernoulli

(pet + q)

(peit + q)

Binomial

(pet + q)n

(peit + q)n

Poisson
Geometric

pet /(1 qet )

etb eta
t(ba)

Uniform over (a, b)

Normal
Exponential

(et1 )

peit /(1 qeit )

eitb eita
it(b a)

itm 12 2 t2

tm+ 12 2 t2

= (1 t )1

(1

(eit 1)

68

it 1
)

INVERSION THEOREM

Integer valued R.V.

Let Y = 0, 1, 2, . . . with prob. f (j) = P (Y = j)
cf (t) = E(eitY ) =

eitj f (j)

Inversion Formula:
1
f (k) =
2

eikt (t)dt

69

1
f (k) =
2
Z

1
eikt
eitj f (j) dt =
2

j=

i(jk)t

dt

j=

ei(jk)t dt

[cos(j k)t + i sin(j k)t]dt




 sin(j k)t
cos(j k)t


+ i
for j 6= k

k
(j

k)

sin n = 0 for any integer n.

cos n = cos(n)
Z

f (j)

ei(jk)t

0 for j 6= k
dt =
2 for j = k
70

INVERSION FORMULA FOR CONTINUOUS

TYPE RANDOM VARIABLES

Let
Z Y have pdf f (y) and cf (t) which is integrable; i.e.
|(t)| dt <

Inversion Formulae:
1
f (y) =
2

eiyt (t)dt

Ex. Normal Distribution (standard normal)

Z
y 2 /2
e
ity
t2 /2

dy = e
e
(t) =
2

71

Replace t by t

y /2
2
ity e

dy = et /2
e
2

Interchange symbols t and y

Z
t2 /2
ity e
y 2 /2

dt = e
e
2

Divide by

2
1
2

eity et

y /2
e
/2
dt =
2

which is the inversion formular for N (0, 1).

72

UNIQUENESS THEOREM

Let X be an arbitrary r.v. and let Y be N (0, 1) where X and Y are

independent. Consider Z = X + cY (c is a constant).
Z (t) = X (t)e

c2 t2 /2

Note that Z (t) is integrable as |X (t)| 1

Z
1
eitz z (t)dt
f (z) =
2

73

P (a < Z b) =
=

FZ (b) FZ (a) =
Z

b
a

1
2

1
2
Z

f (z)dz
a

it(x+cy)

x (t)e

c2 t2 /2

dtdz

"Z

itx

dx x (t)e

c2 t2 /2 itcy

Let c 0 z x and dz dx
Fx (b) Fx (a) =

1
2

ibt

e
it

iat

x (t)dt

The distribution function is determined by its c.f.

If two r.v.s have same characteristic function they have the same
distribution function.
74

dt

Examples
Double Exponential f (x) = 12 e|x|
<x<
Z
Z
1
1
eitx e|x| dx =
[cos tx + i sin tx]e|x| dx
(t) =
2
2

Since sin(tx) = sin tx and cos(tx) = cos(tx)

Z
Z
1
(cos tx)e|x| dx =
(cos tx)ex dx
(t) =
2
0

as sin 0 = 0,


 ex [t sin tx cos tx]
1

=
(t) = 
2
1
+
t
1 + t2
0
cos 0 = 1

Double Exponential is sometimes called Laplaces Distribution.

75

Ex. What is c.f. of the Cauchy distribution which has

1
< y < ?
pdf f (y) = (1+y
2)
Note that by the inversion formula for the double exponential
Z
1
1
1 |y|
ity
e =
e
dt
2
2
1 + t2
Hence for the Cauchy distribution
Z
eity
(t) =

1
|t|
dy
=
e
(1 + y 2 )

76

PROBLEMS

1. Let Y be any r.v.

(a) Show y (t) = E[cos tY ] + iE[sin tY ]
(b) Show y (t) = E[cos tY ] iE[sin tY ]
(c) Show y (t) = y (t)

2. Let Y have a symmetric distribution around 0, i.e. f (y) = f (y)

(a) Show E(sin tY ) = 0 and y (t) is real valued.
(b) Show y (t) = y (t)

3. Let X and Y be ind. ident. dist. r.v.s. Show XY (t) = |x (t)|

77

4. Let Yj (j = 1, 2) be independent exponential r.v.s with

E(Yj ) = 1/j .
Consider S = Y1 + Y2
(a) Find the c.f. of S
(b) Using (i) and the inversion theorem find the pdf of S. Hint: If
f (y) is exponential with parameter (E(Y ) = 1/), then
R ity

1
y
dt
=
e
by the inversion
(t) = it and 2 e
it
formula.

78

2.7 Notes on Partial Fractions

Suppose N (s) and D (s) are polynomials in s. Suppose D (s) is a
polynomial of degree k and N (s) has degree < k.
D (s) =

k
Y

i=1

(s si ) (Roots are distinct)

and roots of N (s) do not coincide with D (s) . Then

k

N (s) X Ai
=

D (s)
s si
i=1
where constants Ai are to be determined.
Since

L {et } = ( + s)1 , et = L1 {( + s)1 }

  X
k
(s)
N
si t
=
L1
A
e
i
D (s)
i=1
79

N (s)
D (s)

N (s)
D (s)

N (s) =

k
X
Ai
s si
i=1

Pk

i=1

k
X

Ai
Qk

1 (s

Ai

i=1

Y
j6=i

j6=i (s

si )

sj )

(s sj )

Substitute s = sr

N (sr ) = Ar

j6=r

N (sr )
Ar = Q
j6=r (sr sj )

(sr sj )
r = 1, 2, . . . , k

80

Since
D (s)

k
Y

i=1

dD (s)
ds
0

D (sr )

(s si )

D0 (s) =

k Y
X
i=1 j6=i

j6=r

(s sj )

(sr sj )

N (sr )
Ar = 0
D (sr )

r = 1, 2, . . . , k

81

Example:
Let Ti be independent r.v. having pdf qi (t) = i ei t (1 6= 2 ) and
consider T = T1 + T2 .
qi (s)

i /(i + s) i = 1, 2

qT (s)

q1 (s)q2 (s) = 1 2 /(1 + s)(2 + s)

A2
A1
+
(1 + s) (2 + s)

N (s) = 1 2

D (s) = (1 + s)(2 + s), si = i

N (sr )
Ar = 0
D (sr )

D0 (s) = (1 + s) + (2 + s)

1 2
2 1

A2 =

A1 =

1 2
1 2
82




1
1 2
qT (s) =

2 1 s + 1
s + 2

Taking inverse transforms gives


1 2  1 t
2 t
q(t) =
e
e
2 1

83

Multiple Roots
If D (s) has multiple roots, the methods for finding the partial fraction
decomposition is more complicated. It is easier to use direct methods.
Example: N (s) = 1, D (s) = (a + s)2 (b + s)
1
A1
A2
B
N
=
=
+
+
D (s)
(a + s)2 (b + s)
a + s (a + s)2
b+s
A1 (a + s)(b + s) + A2 (b + s) + B(a + s)2
1
=
2
(a + s) (b + s)
(a + s)2 (b + s)
1 = A1 (a + s)(b + s) + A2 (b + s) + B(a + s)2

(1)

1 = s2 [A1 + B] + s[A1 (a + b) + A2 + B(2a)] + [abA1 + A2 b + Ba2 ]

A1 + B = 0, A1 (a + b) + A2 + 2aB = 0
A1 (ab) + A2 b + Ba2 = 1
84

The above three equations result in solutions for (A1 , A2 , B). Another
way to solve for these constants is by substituting s = a and s = b in
(1); i.e.
substituting s = b in (1)
B = (a b)2 A1 = B = (a b)2
substituting s = a in (1)

A2 = (b a)1

A1 = (a b)2 , A2 = (a b)1 , B = (a b)2

1
A1
A2
B
N (s)
=
=
+
+
D (s)
(s + a)2 (s + b)
s + a (s + a)2
s+b
 
N (s)
2 at
1 at
2 bt
=
(a

b)
e

(a

b)
te
+
(a

b)
e
L1

D (s)
ebt
eat
=
[1 + t(a b)] +
2
(a b)
(a b)2
85

In general if D (s) = (s s1 )d1 (s s2 )d2 ...(s sk )dk the

decomposition is
A1
A2
Ad 1
N (s)
=
+
+
.
.
.
+
D (s)
s s1
(s s1 )2
(s s1 )d1
+

B1
B2
Bd 2
+
+
.
.
.
+
s s2
(s s2 )2
(s s2 )d2

+ ... . +

Z2
Zdk
Z1
+
+
.
.
.
+
(s sk ) (s sk )2
(s sk )dk

86