1

Chapter 1 Axioms of Probability

C.M. Liu
Feb. 19, 2011
2
Contents
Introduction
Sample Space and Events
Axioms of Probability
Basic Theorems.
Continuity of Probability Function
Probabilities 0 and 1
Random Selection of Points from Intervals
3
Introduction
Ancient Egypians, 3500 B.C.
Use a four-sided die-shaped bone.
Hounds and Jackals.
Studies of chances of events in 15th century
Luca Paccioli (1445 1514)
Niccolo Tartaglia (1499 1557)
Girolamo Cardano (1501 1576)
Galileo Galilei (1564 1642)
4
Introduction
Real Progress from 1654
Blaise Pascal (1623 1662).
Pierre de Fermat (1601 1665).
Christian Huygens (1629 1695).
On Calculations in Games of
Chance
Major Breakthrough
James Bernoulli (1654 1705).
Abraham de Moivre (1667
1754).
Pascal
Fermat
Moivre
Bernoulli
Christiaan Huygens
5
Introduction
18th century
Laplace, Poisson, Gauss expanded the growth of probability
and its application.
19th Century
Advanced the works to put it on firm mathematical grounds.
Pafnuty Chebyshev
Andrei Markov.
Aleksandre Lyapunov.
20th Century
David Hilbert (1862 1943)
23 problems whose solutions were crucial to the advancement
of mathematics.
Andrei Kolmogorov (1903-1987)
Combined the notion of sample space, introduced by Richard von
Mises, and measure theory and presented his axiom system for
probability theory in 1933.
Laplace
Kolmogorov
6
2. Sample Space and Events
Basic definitions
Sample Space, S: The set of all possible
outcomes.
Sample Points: The outcomes.
Event: A subset of the sample space.
7
2. Sample Space and Events
Experiment (eg. Tossing a die)
Outcome(sample point)
Sample space={all outcomes}
Event: subset of sample space
Ex1.1 tossing a coin once
sample space S = {H, T}
Ex1.2 flipping a coin and tossing a die if T
or flipping a coin again if H
S={T1,T2,T3,T4,T5,T6,HT,HH}
8
2. Sample Space and Events
Ex1.3 measuring the lifetime of a light bulb
S={x: x 0}
E={x: x 100} is the event that the light
bulb lasts at least 100 hours
Ex1.4 all families with 1, 2, or 3 children
(genders specified)
S={b,g,bg,gb,bb,gg,bbb,bgb,bbg,bgg,
ggg,gbg,ggb,gbb}
9
2. Sample Space and Events--
Subset Relationship
Subset
EcFxeExeF
Equal
E=FEcF and FcE
Intersection
The intersection of E and F, written E F, is the set of
elements that belong to both E and F.
EF=EF={x: xeE and xeF}
Union
The union of E and F, written E F, is the set of elements
that belong to either E or F.
EF={x: xeE or xeF}.
10
2. Sample Space and Events--
Subset Relationship
Complement
The complement of E, written E
c
, is the set of all
elements that are not in E.
E
c
={x: xeE}.
Difference of Two Events.
The set of elements belong to E but not in F.
E-F={x: xeE and xeF}.
Mutually Exclusive.
The joint occurrence of any two event is
impossible.
EF= |
11
2. Sample Space and Events--
Subset Relationship
Venn Diagrams
EF EF
E
c
(E
c
G)F
E F
E F
E
G
E F
Intuitive justification, create counter-
examples, and shows invalidity.
12
2. Sample Space and Events--
Subset Relationship
For any three events A, B, and C defined
on a sample space S,
Commutativity: EF=FE, EF=FE.
Associativity: E(FG)=(EF)G, E(FG)=(EF)G
Distributative Laws: (EF)H =(EH)(FH)
(EF)H=(EH)(FH),
DeMorgans Laws:
(EF)
c
=E
c
F
c
,
(EF)
c
=E
c
F
c

n
i
c
i
c
n
i
i
E E
0 0 = =
=
|
|
.
|

\
|

=
=
|
|
.
|

\
|
1 1 i
c
i
c
i
i
E E

n
i
c
i
c
n
i
i
E E
0 0 = =
=
|
|
.
|

\
|

=
=
|
|
.
|

\
|
1 1 i
c
i
c
i
i
E E
13
Proof of DeMorgans Laws
c c c
F E F E _ ) (
c c c
F E F E _ ) (
c
F E x Let ) ( e
) ( F E x Then e
F x and E x So e e ,
) ( ,
c c
F E x Hence e
c c
F E x Let e
c c
F x and E x Then e e
F x and E x So e e ,
F E x Therefore e ,
c
F E x Thus ) ( , e
14
3. Axioms of Probability
Axioms of Probability:
Let S be the sample space of a random phenomenon.
Suppose that to each event A of S. a number denoted by
P(A), is associated with A. If P satisfies the following axioms,
then it is called a probability and the number P(A) is said to
be the probability of A.
P(A) > 0 for any event A.
P(S) = 1 where S is the sample space.
If {A
i
}, i=1,2,, is a sequence of mutually exclusive
events (that is, A
i
A
j
=| for all i=j), then

=
=
1
1
) ( ) (
i
i i
i
A P A P
15
3. Axioms of Probability
Theorem 1.1
The probability of the empty set P(|)=0.
16
3. Axioms of Probability
Theorem 1.2
If {A
i
}, i=1,2,n, are mutually exclusive (that
is, A
i
A
j
=| for all i=j), then

=
=
=
n
i
i i
n
i
A P A P
1
1
) ( ) (
17
Examples
Flipping a fair or unbiased Coin
Events
Sample space, S
Probability on sample space and events.
Probability on unbiased coin.
Probability on biased coin.
18
3. Axioms of Probability
Theorem 1.3
Let S be the sample space of an experiment. If
S has N points that are all equally likely occur,
then for any event A of S,
where N(A) is the number of points of A.
Example
Flipping a fair coin three times and A be the
event at least two heads.
N
A N
A P
) (
) ( =
19
Proof on Theorem 1.3
20
Examples
21
Solutions
22
4. Basic Theorems
Theorem 1.4
Theorem 1.5
Corollary
then B A If , _
) ( ) ( ) ( ) ( A P B P BA P A B P
c
= =
). ( 1 ) ( , A P A P A event any For
c
=
) ( ) ( , B P A P then B A If s _
23
Proof
24
4. Basic Theorems
Theorem 1.6
Inclusion-Exclusion Principle
Theorem 1.7
). ( ) ( ) ( ) ( AB P B P A P B A P + =
). ( ) 1 ( ... ) (
) ( ) ( ) (
2 1
1
2
1
1
1 1
1
1 1 1 1
n
n
n
i
n
i j
n
j k
k j i
n
i
n
i j
j i
n
i
i
n
i
i
A A A P A A A P
A A P A P A P
+ +
=

+ = + =

= + = = =


) ( ) ( ) (
c
AB P AB P A P + =
25
4. Basic Theorems
S
A B
26
4. Basic Theorems
Ex 1.15 In a community of 400 adults, 300 bike or
swim or do both, 160 swim, and 120 swim and bike.
What is the probability that an adult, selected at
random from this community, bike?
Sol: A: event that the person swims
B: event that the person bikes
P(AUB)=300/400, P(A)=160/400,
P(AB)=120/400
P(B)=P(AUB)+P(AB)-P(A)
= 300/400+120/400-160/400=260/400= 0.65
27
4. Basic Theorems
Ex 1.16 A number is chosen at random from the
set of numbers {1, 2, 3, , 1000}. What is the
probability that it is divisible by 3 or 5(I.e. either
3 or 5 or both)?
Sol: A: event that the outcome is divisible by 3
B: event that the outcome is divisible by 5
P(AUB)=P(A)+P(B)-P(AB)
=333/1000+200/1000-66/1000
=467/1000
28
4. Basic Theorems
Inclusion-Exclusion Principle
29
4. Basic Theorems
Inclusion-Exclusion Principle
) ... ( ) 1 ( ... ) (
) ( ) ( ) ... (
2 1
1
2 1
n
n
k j i
j i i n
A A A P A A A P
A A P A P A A A P

+ +
=

30
4. Basic Theorems-- Examples
31
4. Basic Theorems-- Examples
(c.1)
32
4. Basic Theorems
Theorem 1.7 P(A) = P(AB) + P(AB
c
)
Proof:
) ( ) ( ) ( ) (
) (
c c
c
c c
AB P AB P AB AB P A P
exclusive mutually are AB and AB Since
AB AB B B A AS A
+ = =
= = =
33
Examples (c.2)
34
Continuous Functions
Let R denotes the set of all real numbers.
is called continuous at a point if
It is called continuous on R if it is continuous at all points.
Sequential Criterion
f(x) is continuous on R if and only if, for every convergent
sequence in R.
5. Continuity of Probability Functions
. : R R f
R ce
) ( ) ( lim c f x f
c x
=

) lim ( ) ( lim
n
n
n
n
x f x f

=
R ce

=1
} {
n n
x
35
5. Continuity of Probability Functions
Increasing Sequence of Events of Sample Space

Decreasing Sequence of Events of Sample Space

For increasing events

For increasing sequence of events, means the event that
at least one E
i
occurs
For decreasing sequence of events, means the event that
every E
i
occurs
_ _ _ _
+1 2 1 n n
E E E E
_ _ _ _
+1 2 1 n n
E E E E
n
n
n
n
E E

=
=
1
lim
n
n
n
n
E E

=
=
1
lim
Applicable to the probability
density function ?
n
n
E

lim
n
n
E

lim
36
5. Continuity of Probability Functions
Theorem 1.8 Continuity of Probability Function
For any increasing or decreasing sequence of events,
{ }. lim ) ( lim
n
n
n
n
E P E P

=
37
Example
38
6. Probabilities 0 and 1
Not correct speculation
If E and F are events with probabilities 1 and 0, respectively, it
is not correct to say that E is the sample space and F is the
empty space.
Ex. P(1/3) in (0, 1).
39
7. Random Selection of Points
from Intervals
Randomly selected from an Interval
A point is said to be randomly selected from an interval
(a, b) if any two subintervals of (a, b) that have the same
length are equally likely to include the point. The probability
associated with the event that the subinterval (o, |) contains
the point is defined to be (|-o)/(b-a).