2 Random Variables
Definition 1 (Probability Space) A probability space is a tuple (Ω, P), where P : Ω → [0, 1]
is a probability distribution.
Definition 2 (Random Variable) A random variable is a tuple ((Ω, P), X, S), where (Ω, P) is
a probability space and X : Ω → S is a function. We say X is a random variable taking values
in S.
Note Unless stated otherwise, we shall consider random variables with finite range, i.e., S is a
finite set.
P PX
0, 1 0, 1
P
It is easy to check that PX is indeed a probability distribution, i.e., X (s) = 1. The
s PP
collection {X −1 (s) | s ∈ S} forms a partition of Ω. Therefore s PX (s) = −1 (s)] =
P
s P[X
−1
P[∪s∈S X (s)] = P(Ω) = 1.
Fact 2 Consider a random variable ((Ω, P), X, S). Let E be an experiment such that E ∼ P.
Then EX ∼ PX , where the experiment EX is described as follows
1. Obtain w ← Run(E)
2. Output X(w)
Experiment EX
1
Example 1 Consider the uniform probability distribution associated with the experiment of
rolling a dice. Let X be the random variable that associates even to the sample points 2, 4
and 6, and odd to 1, 3 and 5.
X
Ω = {1, 2, 3, 4, 5, 6} S = {even, odd}
P PX
0, 1 0, 1
Then the probability distribution PX : {even, odd} → [0, 1] of the random variable X is
computed as follows. PX (even) = P[X −1 (even)] = P[{2, 4, 6}] = 36 = 12 . Similarly, PX (odd) =
P[X −1 (odd)] = P[{1, 3, 5}] = 36 = 21 .
Notation: We sometime denote a random variable ((Ω, P), X, S) simply as (X, S).
• S = {0, 1, . . . , n}
• 0≤p≤1
n
pi q n−i , where q = 1 − p.
• for i ∈ S, PX [i] = i
The probability distribution P(X,Y ) is called the joint probability distribution of X and Y .
P
Exercise 1 Check that, (s,t)∈S×T P(X,Y ) (s, t) = 1.
P(X,Y ) (s, t) = P[X −1 (s) ∩ Y −1 (t)] = P[X −1 (s)] × P[Y −1 (t)] = PX (s) × PY (t).
2
Marginal Distributions
Suppose X and Y are two random variables taking values in S and T respectively. Suppose
you are given the joint probability distribution P(X,Y ) of X and Y .
For s ∈ S,
X f
Ω S T
P PX Pf (X)
0, 1 0, 1 0, 1
P
Exercise 2 Check that, t∈T Pf (X) (t) = 1.
Practice Problems
1. Two 4-sided unbiased dice are thrown. Let X be the score on the first dice and Y be the
larger of two scores. Write down the joint distribution of X and Y .
3
2. A contractor purchases a shipment of 200 phones. It is his policy to test 10 random pieces
of these phones and to keep the shipment if at least 9 of the 10 are in working condition.
If the shipment contains 40 defective phones, what is the probability it will be kept?
4. Let X : Ω → S and Y : Ω → T be two random variables such that the joint random
varible (X, Y ) is uniform over S × T . Determine the distributions X and Y . Are X and Y
independent ?
Hint: The joint random variable (Xi , Xi ) taking a value (s, t) captures the event that ball
i lands in bin s and ball j lands in bin t.
6. Suppose g : S → T is a onto function such that the number of pre-images under g of every
t ∈ T is the same, i.e., there exists a k ∈ N such that |g −1 (t)| = k for all t ∈ T . Let X
be a random variable that is uniformly distributed over S. Show that the random variable
g(X) is uniformly distributed over T .
7. Let X be a random variable taking values in the set {1, . . . , m}. Let Y be a uniform random
variable taking values in the set {PX (1), . . . , PX (m)}. Then
Pm 2
(a) i=1 PX (i) ≥ 1/m
Pm 2 m+1
(b) i=1 PX (i) = 2
Pm 2
(c) i=1 PX (i) ≤ 1/m
Pm 2
(d) i=1 PX (i) = 1/m
8. Let P : {0, 1}3 → [0, 1] is uniform probability distribution. Let X : {0, 1}3 → {0, 1} be a
random variable that maps elements (a3 a2 a1 ) ∈ {0, 1}3 to 1 if and only if a3 = a1 . Let
Y : {0, 1}3 → {0, 1} be a random variable that maps elements (a3 a2 a1 ) ∈ {0, 1}3 to 1 if
and only if a3 + a2 + a1 = 2. Then P(X,Y ) (1, 1) is 1/4 or 3/8 or 1/2 or 0.
9. Consider the joint random variable (X, Y ) taking values in {0, 1}×{0, 1, 2}. It is given that,
for any (a, b) ∈ {0, 1} × {0, 1, 2}, P(X,Y ) = a+b
9 . Then PX (1) is 1/9 or 1/3 or 2/3 or 1/6.
where k is a real constant. Then PX (0) equals 1/9 or 2/27 or 1/27 or 1/81. [JAM]
4
1 1
Let X ∼ Bin[3,
11. " # 3 ], Y ∼ Bin[3, 3 ] and X, Y are independent. Compute the probability that
X
√ √Y
2 2 is orthogonal. [JAM]
− √12 √1
2