Probability: Chapter 1

Experiments, Models, and Probabilities

Section 1.1: Set Theory

Bernard Fulgham
University of Virginia

January 20, 2017

1 / 39 2 / 39

Sets / Slide 1 Sets / Slide 2

Notation
Usually capital letters for sets: A, B, C.
Union: x ∈ A ∪ B ⇐⇒ x ∈ A or x ∈ B.
Lower case for elements of sets: x, y, z.
x ∈ A means “x is an element of A” Intersection: x ∈ A ∩ B ⇐⇒ x ∈ A and x ∈ B.
x∈/ A means “x is not an element of A” Universal set: set of all integers, all galaxies, all bulldogs, etc.
B = {x 2 | x = 1, 2, 3} = {1, 4, 9} Complement: x ∈ Ac ⇐⇒ x ∈
/ A.

Subsets: Difference: x ∈ A \ B ⇐⇒ x ∈ A and x ∈

/ B;
A ⊂ B ⇐⇒ x ∈ A =⇒ x ∈ B,
A \ B = A ∩ Bc and Ac = S \ A.
A = B ⇐⇒ A ⊂ B and B ⊂ A.
Null set: Disjoint: A ∩ B = ∅.
∅ = { } a.k.a. empty set,
∅ ⊂ A for any set A.

3 / 39 4 / 39
Definitions Partition

A collection of sets A1 , A2 , . . . , An is: A partition is a collection of sets that is both:

I mutually exclusive (ME) if mutually exclusive and collectively exhaustive.

Ai ∩ Aj = ∅ for i 6= j; Example
For i = 0, 1, 2, 3, 4, Bi = {outcomes with i heads}.

I collectively exhaustive (CE) if Each Bi is an event containing one or more outcomes:

A1 ∪ A2 ∪ · · · ∪ An = S. B0 = {tttt}, B1 = {httt, thtt, ttht, ttth}, etc.

Is the set B = {B0 , B1 , B2 , B3 , B4 } a partition?

5 / 39 6 / 39

Experiments

Procedure + Observations
Real Experiments are too complicated.
Instead, we analyze/develop models of experiments.
Section 1.2: Applying Set Theory to Probability
Example An experiment consists of the following procedure,
observation and model:
1. Procedure: Flip a coin and let it land on a table.
2. Observation: Observe which side (head or tail) faces you after
the coin lands.
3. Model: Heads and tails are equally likely. The result of each
flip is unrelated to the results of previous flips.

7 / 39 8 / 39
Outcomes Example

An outcome of an experiment is any possible observation of that

experiment.
The sample space of an experiment is the finest-grain, mutually
exclusive, collectively exhaustive set of all possible outcomes.
An event is any set of outcomes of an experiment.
Flip four coins: a penny, a nickel, a dime, and a quarter. Examine
the coins in order (penny, then nickel, then dime, then quarter)
and observe whether each coin shows a head (h) or a tail (t).
What is the sample space?
How many elements are in the sample space?

Set Theory Probability The sample space consists of 16 four-letter words:

set event
{tttt, ttth, ttht, . . . , hhhh}.
universal set sample space
element outcome

9 / 39 10 / 39

Theorem

For a partition B = {B1 , B2 , . . .} and any event A, define

Ci = A ∩ Bi . Then Section 1.3: Probability Axioms
I the events Ci ∩ Cj = ∅ for i 6= j, and
I A = C1 ∪ C2 ∪ · · · .

11 / 39 12 / 39
Axioms Consequences

Theorem 1.3:
If A = A1 ∪ A2 ∪ · · · ∪ Am and Ai ∩ Aj = ∅ for i 6= j, then
A probability measure P[·] is a function that maps events in the
m
sample space to real numbers such that X
P[A] = P[Ai ].
Axiom 1 For any event A, P[A] ≥ 0. i=1

Axiom 2 P[S] = 1 Theorem 1.4:

Axiom 3 For any countable collection A1 , A2 , . . . of mutually The probability measure P[·] satisfies the following:
exclusive events, (a) P[∅] = 0
P[A1 ∪ A2 ∪ · · · ] = P[A1 ] + P[A2 ] + · · · (b) P[Ac ] = 1 − P[A]
(c) For any events A and B,
P[A ∪ B] = P[A] + P[B] − P[A ∩ B].

(d) If A ⊂ B, then P[A] ≤ P[B].

13 / 39 14 / 39

Example 1 Example 2 / Slide 1

A student’s score on a 10-point quiz is equally likely to be any
integer between 2 and 10.
What is the probability of an A, which requires the student to get
a score of 9 or more?
A phone call was made by Veronica (V ) or Doug (D), and it is
long (L) if the duration is more than 30 minutes; otherwise, it is
brief (B).

If G is the grade and N is the score, then Given: P[V ] = 0.7 P[L] = 0.6 P[VL] = 0.35
B L
1 1 2
P[G = A] = P[N ≥ 9] = P[N = 9] + P[N = 10] = + = (a) Fill in the following probability table: V
9 9 9
D

What is the probability the student earns an F by scoring less than B L

4? Answer: V 0.35 0.35
D 0.05 0.25
1 1 2
P[G = F ] = P[N < 4] = P[N = 2] + P[N = 3] = + =
9 9 9

15 / 39 16 / 39
Example 2 / Slide 2
(b) Compute the following:

P[DL] P[D ∪ L] P[VB]

P[V ∪ L] P[V ∪ D] P[LB]

B L Section 1.4: Conditional Probability

V 0.35 0.35
D 0.05 0.25

P[DL] = P[D ∩ L] = 0.25

P[VB] = P[V ∩ B] = 0.35
P[D ∪ L] = 1 − P[V ∩ B] = 1 − 0.35 = 0.65
P[V ∪ L] = 1 − P[D ∩ B] = 1 − 0.05 = 0.95
P[LB] = P[L ∩ B] = 0 P[V ∪ D] = 1

17 / 39 18 / 39

Conditioning Conditional Probability

Learning that event B occurred changes the likelihood of A.

P[A] is our knowledge of the likelihood of A. The conditional probability of A given the occurrence of B is
P[A] is the a priori probability, i.e., it reflects our knowledge of the P[A B]
likelihood of A prior to performing an experiment. P[A | B] = , provided that P[B] 6= 0.
P[B]
Suppose we cannot completely observe an experiment:
We learn that event B occurred. However, we do not learn the Note the following:
precise outcome.
P[A B] = P[A | B] P[B] = P[B | A] P[A].

19 / 39 20 / 39
 Law of Total Probability

If B1 , B2 , . . . , Bn is a partition and P[Bi ] > 0 for i = 1, . . . , n, then

Section 1.5: The Law of Total Probability
n
X n
X
P[A] = P[A ∩ Bi ] = P[A | Bi ] P[Bi ].
i=1 i=1

21 / 39 22 / 39

Bayes’ Theorem Sequential Experiment: Example

Two coins: one biased, one fair, but you don’t know which is
P[A B] P[A | B] P[B] which.
P[B | A] = =
P[A] P[A]
Coin 1: P[H] = 3/4 Coin 2: P[H] = 1/2
If B1 , B2 , . . . , Bn , is a partition, then Pick a coin at random and flip it.
Let Ci denote the event that coin i is picked.
P[A | Bi ] P[Bi ]
P[Bi | A] = Pn .
i=1 P[A | Bi ] P[Bi ] What is P[C1 | H], the probability that the biased coin was flipped,
given that the coin flip was heads?

23 / 39 24 / 39
Solution: Tree Diagram Which Coin was Flipped?

3/4 H
P[C1 H]
C1 P[C1 | H] = P[H | C1 ] = 3/4, P[H | C2 ] = 1/2 =⇒ P[C1 | H] = 3/5
P[H]
1/2
1/4
T Now compute: P[C2 | T ].
P[C1 H]
= If the flip is heads, what is your guess for which coin was flipped?
P[C1 H] + P[C2 H]
1/2 H If the flip is tails, what is your guess?
1/2 3/8 3
C2 = = Finally, compute the probability that you guess the correct coin.
3/8 + 1/4 5
1/2
T

25 / 39 26 / 39

Solution: Part I Solution: Part II

1 3 3
3/4
H P[C1 H] = 2 · 4 = 8

C1 P[C2 T ] P[C2 T ] 1/4 2

P[C2 | T ] = = = =
1/2 P[T ] 1 − P[H] 1 − 5/8 3
1/4 1 1 1
T P[C1 T ] = 2 · 4 = 8
5
P[C1 | H] = =⇒ guess Coin 1 if flip is Heads
8
1 1 1
1/2
H P[C2 H] = 2 · 2 = 4 2
1/2 P[C2 | T ] = =⇒ guess Coin 2 if flip is Tails
3
C2
3 1 5
1/2
T P[C2 T ] = 1
· 1
= 1 P[Correct] = P[C1 H] + P[C2 T ] = + =
2 2 4 8 4 8

27 / 39 28 / 39
Bayes’ Theorem: Test for a Rare Disease Test for a Rare Disease: Tree Diagram

A disease is carried by 1 in 500. Assume that the rates of false

positives and false negatives are both 1%.
0.99
+
D : a person carries the disease.
D
N : a person is not a carrier.
0.002 0.01
1. Compute P[− | D], the probability that the test is negative, −
given that the person carries the disease.
2. Compute P[+ | N], the probability that the test is positive, +
0.998 0.01
given that the person isn’t a carrier.
N
3. Compute P[D | +], the probability that the person is a carrier,
given that the test is positive. 0.99
−
4. Compute P[test is correct].

29 / 39 30 / 39

Test for a Rare Disease: Solution Bayes’ Theorem: Drug Testing

We’re interested in the efficacy of a drug test. For a single test, D
is the event that the person tested is using the drug, while N is the
P[D, +] event that the person is not a user.
P[D | +] =
P[+] I The sensitivity of the test is measured by P[+ | D]. For this
(0.002) · (0.99) test, P[+ | D] = 0.99 (99% sensitive).
=
(0.002) · (0.99) + (0.998) · (0.01) Note that P[+ | D] = 0.99 =⇒ P[− | D] = 0.01. This is the
≈ 0.166 probability of a false negative.
I The specificity of the test is measured by P[− | N]. In this
P[Correct] = P[D, +] + P[N, −] case, P[− | N] = 0.99 (99% specific).
Note that P[− | N] = 0.99 =⇒ P[+ | N] = 0.01. This is the
= (0.002) · (0.99) + (0.998) · (0.99)
probability of a false positive.
= 0.99 I Only 0.5% use the drug. That is, P[D] = 0.005.
Compute P[D | +], the probability that a positive test accurately
identifies a user.

31 / 39 32 / 39
Drug Testing: Tree Diagram Drug Testing: Solution

+ P[D, +]
0.99 P[D | +] =
P[+]
D (0.005) · (0.99)
0.005 =
0.01
− (0.005) · (0.99) + (0.995) · (0.01)
≈ 0.332

+
0.995 0.01 P[Correct] = P[D, +] + P[N, −]
N = (0.005) · (0.99) + (0.995) · (0.99)
0.99
− = 0.99

33 / 39 34 / 39

Independence / Slide 1

Events A and B are independent if and only if

P[AB] = P[A] P[B].
If P[A], P[B] 6= 0, then independence is equivalent to:
Section 1.6: Independence P[A | B] = P[A] and P[B | A] = P[B].
Knowing that B occurred doesn’t alter the probability of A
happening, and vice versa.
Note: In most situations, independent events are not disjoint.
Exceptions occur only when P[A] = 0 or P[B] = 0.
A ∩ B = ∅ =⇒ P[A ∪ B] = P[A] + P[B], but
A and B independent =⇒ P[A ∩ B] = P[A] · P[B].

35 / 39 36 / 39
Independence / Slide 2 Example

Two consecutive data packets pass through an internet router, and

each contains either video (v ) or data (d). The two data packets
are independent and the probability that a single packet contains
Events A1 , A2 , and A3 are independent if and only if
video is 0.8.
I A1 and A2 are independent,
Ci = ith packet and Nv = number of video packets.
I A2 and A3 are independent,
I A1 and A3 are independent, Are the following events independent?
I P[A1 ∩ A2 ∩ A3 ] = P[A1 ] P[A2 ] P[A3 ]. 1. {Nv = 2} and {Nv ≥ 1}
2. {Nv ≥ 1} and C1 = v
3. C1 = d and C2 = v
4. C2 = v and {Nv is even}

37 / 39 38 / 39

Solution

0.8 v P[C1 = v ] = P[C2 = v ] = 0.8

v
0.8 0.2
P[C1 = d] = P[C2 = d] = 0.2
d
P[Nv = 2] = P[vv ] = (0.8)2 = 0.64
0.8 v
0.2
d
0.2
d

P[Nv ≥ 1] = 1 − P[Nv = 0] = 1 − P[dd] = 1 − (0.2)2 = 0.96

P[Nv is even] = P[Nv = 0] + P[Nv = 2] = P[dd] + P[vv ]

= (0.2)2 + (0.8)2 = 0.68

39 / 39