Chapter 5
February 19, 2014
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 2
Marginal Probability Mass Function
x =, y = 0 1 2 3 p(x)
0 0.08 0.07 0.04 0.00 0.19
1 0.06 0.15 0.05 0.04 0.30
2 0.05 0.04 0.10 0.06 0.25
3 0.00 0.03 0.04 0.07 0.14
4 0.00 0.01 0.05 0.06 0.12
p(y) 0.19 0.30 0.28 0.23 1.00
Definition 3. Let X and Y be continuous rvs. Then f (x, y) is the joint probability
density function for X and Y if for any two-dimensional set A:
Z Z
P [(X, Y ) A] = f (x, y)dxdy
A
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 3
Marginal Probability Density Function
Definition 4. The marginal probability density function of X and Y , denoted
fX (x) and fY (y), respectively, are given by:
Z
fX (x) = f (x, y)dy, < X < .
Z
fY (y) = f (x, y)dx, < Y < .
Example of Independence
Example 5.1.2 X follows an exponential distribution with = 2, Y follows an
exponential distribution with = 3, X and Y are independent, find f (x, y).
f (x, y) = fX (x) fY (y) = 2e2x 3e3y = 6e(2x+3y) , x 0, y 0
Example 5.1.3 Toss a fair coin, and a die. Let X = 1 if coin is head, let X = 0 if coin
is tail. Let Y be the outcome of the die. if X and Y are independent, find p(x, y) and
find the probability that the outcome of the die is greater than 3 and the coin is a head?
x=,y= 1 2 3 4 5 6
1
0 2 16 1
2 16 1
2 16 1
2 16 1
2 16 1
2 16
1
1 2 16 1
2 16 1
2 16 1
2 16 1
2 16 1
2 16
P (X = 1, Y > 3) = P (X = 1) P (Y > 3)
Examples Continued
Example 5.1.4 Given the following p(x, y), is X and Y independent?
x =, y = 0 1 2 3 p(x)
0 0.04 0.03 0.01 0.02 0.1
1 0.08 0.06 0.02 0.04 0.2
2 0.16 0.12 0.04 0.08 0.4
3 0.04 0.03 0.01 0.02 0.1
4 0.08 0.06 0.02 0.04 0.2
p(y) 0.4 0.3 0.1 0.2 1.00
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 4
Example 5.1.5 Give fX (x) = 0.5x, 0 < x < 2, fY (y) = 3y 2 , 0 < y < 1, f (x, y) =
1.5xy 2 , 0 < x < 2 and 0 < y < 1, is X, Y independent?
P (a1 X1 b1 , , an Xn bn )
Z b1 Z bn
= f (x1 , , xn )dxn dx1
a1 an
of the variables, the joint pmf or pdf of the subset is equal to the product of the marginal
pmfs or pdfs.
Conditional Distributions
Definition 6. Let X, Y be two continuous rvs with joint pdf f (x, y) and marginal
pdfs fX (x) and fY (y). Then for any X value x for which fX (x) > 0, the conditional
probability density function of Y given that X = x is:
f (x, y)
fY |X (y|x) = , < y < .
fX (x)
If X and Y are discrete, replace pdfs by pmfs in this definition. That then gives
conditional probability mass function of Y when X = x.
x =, y = 0 1 2 3 p(x)
0 0.08 0.07 0.04 0.00 0.19
1 0.06 0.15 0.05 0.04 0.30
2 0.05 0.04 0.10 0.06 0.25
3 0.00 0.03 0.04 0.07 0.14
4 0.00 0.01 0.05 0.06 0.12
p(y) 0.19 0.30 0.28 0.23 1.00
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 5
Example 5.1.6 Given f (x, y) = 65 (x + y 2 ), 0 x 1, 0 < y < 1. fX (x) = 65 x + 25 .
What is the conditional density of Y given X = 0?
6 2
f (0, y) y
fY |X (y|0) = = 52
fX (0) 5
Definition 7. Let X and Y be jointly distributed rvs with pmf p(x, y) or pdf f (x, y)
according to whether the variables are discrete or continuous. Then the expected value
of a function h(X, Y ), denoted E[h(X, Y )] or h(x,y) is:
P P
x y h(x, y) p(x, y), discrete;
h(X, Y) = E [h(X, Y )] =
R R
h(x, y) f (x, y)dxdy, continuous.
Example 5.2.1 The joint pmf is given below. What is E(XY )? What is
E[max(X, Y )]?
p(x, y) y=0 1 10 20
x=0 0.02 0.06 0.02 0.10
1 0.04 0.15 0.20 0.10
10 0.01 0.15 0.14 0.01
Example 5.2.2 Joint pdf of X and Y is: f (x, y) = 4xy, 0 < x < 1, 0 < y < 1.
What is E(XY )?
Covariance
Definition 8. Let E(X) and E(Y ) denote the expectations of rv X and Y . The
covariance between X and Y , denoted Cov(X, Y ) is defined as:
i.e., P P
x y [x E(X)] [y E(Y )] p(x, y), discrete;
=
R R
(x E(X))(y E(Y ))f (x, y)dxdy, continuous.
Cov(X, X) = V ar(X)
Cov(X, Y ) = Cov(Y, X)
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 6
Shortcut formula:
Cov(X, Y ) = E(XY ) E(X)E(Y )
Interpretation of Covariance
Similar to Variance, Covariance is a measure of variation.
Covariance measures how much two random variables vary together.
As opposed to variance: a measure of variation of a single rv.
If two rvs tend to vary together, then the covariance between the two variables
will be positive.
For example, when one of them is above its expected value, then the other
variable tends to be above its expected value as well.
If two rvs vary differently, then the covariance between the two variables will be
negative.
For example, when one of them is above its expected value, the other variable
tends to be below its expected value.
Examples of Covariance
Example 5.2.1 Exercise The joint pmf is given below. Whats Cov(X, Y )?
p(x, y) y=0 1 10 20
x=0 0.02 0.06 0.02 0.10
1 0.04 0.15 0.20 0.10
10 0.01 0.15 0.14 0.01
Example 5.2.2 Joint pdf of X and Y is: f (x, y) = 4xy, 0 < x < 1, 0 < y < 1.
Whats Cov(X, Y )?
Example 5.2.3 Given the pmf below, whats Cov(X, Y )?
Correlation
Definition 9. The correlation coefficient of two rvs X and Y , denoted Corr(X, Y ),
X,Y or just is defined by:
Cov(X, Y )
Corr(X, Y ) = X,Y = p p
V ar(X) V ar(Y )
i.e.,
Cov(X, Y )
Corr(X, Y ) = X,Y =
X Y
where X and Y are the std devs of X and Y , respectively.
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 7
Properties and Shortcut Formula of Correlation
Shortcut formula:
E(XY ) E(X)E(Y )
Corr(X, Y ) = p p
E(X ) (E(X))2 E(Y 2 ) (E(Y ))2
2
Interpretation of Correlation
Correlation is a standardized measure.
Examples of Correlation
Example 5.2.1 Exercise The joint pmf is given below. Find Cov(X, Y ).
p(x, y) y=0 1 10 20
x=0 0.02 0.06 0.02 0.10
1 0.04 0.15 0.20 0.10
10 0.01 0.15 0.14 0.01
Example 5.2.2 Joint pdf of X and Y is: f (x, y) = 4xy, 0 < x < 1, 0 < y < 1.
Find Corr(X, Y ).
p(x, y) y = 1 1 3
1 1 1
x = 1 9 9 9
1 1 1
0 9 9 9
1 1 1
1 9 9 9
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 8
5.3 Statistics and their Distributions
Statistic
Definition 10 (Statistic). A statistic is any quantity whose value can be calculated
from sample data. Or, a statistic is a function of random variables. We denote a statistic
by an uppercase letter; a lowercase letter is used to represent the calculated or observed
value of the statistic.
Examples:
X1 +X2
Two rv X1 and X2 , denote X = 2 , X is a statistic.
2+ Independent RVs
Definition 11 (Joint pmf and pdf for more than two rvs). For n rvs X1 , X2 , , Xn ,
the joint pmf is:
p(x1 , x2 , , xn ) = P (X1 = x1 , X2 , = x2 , , Xn = xn )
Random Samples
Definition 13 (Random Sample). The rvs X1 , X2 , , Xn are said to form a (simple)
random sample of size n if:
1. The Xi s are independent rvs.
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 9
Example 5.3.2 Sample variance Take a random sample of size n from a specific distribution(say
standard normal). X = X1 +X2n++Xn is the random variable sample mean.
(Xi X)2
P
S2 = , is a statistic
n1
If X1 = x1 , X2 = x2 , , Xn = xn ,
(xi x)2
P
2
s = , is a value of the rv S 2
n1
Example 5.3.3
Example 5.20 in textbook. A large automobile service center charges $40, $45, and
$50 for a tune-up of four-, six-, and eight-cylinder cars, respectively. 20% of the tune-ups
are done for four-cylinder cars, 30% for six-cylinder cars and 50% for eight-cylinder cars.
Let X be the service charge for a single tune-up. Then the distribution of X is:
x 40 45 50
p(x) 0.2 0.3 0.5
Now let X1 and X2 be the service charges of two randomly selected tune-ups. Find the
distribution of:
X1 +X2
1. X = 2
Example 5.3.3
x1 x2 p(x1 , x2 ) x s2
40 40 0.04 40 0
40 45 0.06 42.5 12.5
40 50 0.10 45 50
45 40 0.06 42.5 12.5
45 45 0.09 45 0
45 50 0.15 47.5 12.5
50 40 0.10 45 50
50 45 0.15 47.5 12.5
50 50 0.25 50 0
Proposition (Mean and Std Dev of Sample Sum). Let X1 , X2 , ..., Xn be a random
sample from a distribution with mean and standard deviation . Sample sum is To =
X1 + X2 + ... + Xn . Then:
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 10
1. E(To ) = n
2. V ar(To ) = n 2 and To = n
Proposition (Mean and Std Dev of Sample Mean). Let X1 , X2 , ..., Xn be a random
sample from a distribution with mean and standard deviation . Sample mean is
X = X1 +X2n+...+Xn . Then:
1. E X =
2
2. V ar X = n and X = n
2. X N (, n )
Example 5.4.1
Example 5.25. Let X be the time it takes a rat to find its way through a maze.
X N ( = 1.5, 2 = 0.352 ) (in minutes). Suppose five rats are randomly selected.
Let X1 , X2 , , X5 denote their times in the maze. Assume X1 , X2 , , X5 be a
random sample from N ( = 1.5, 2 = 0.352 ). Let total time To = X1 + X2 + + X5 ,
average time X = X1 +X25++X5 . What is the probability that the total time of the 5
rats is between 6 and 8 minutes? What is the probability that the average time is at
most 2.0 minutes?
0.352
X N (1.5, )
5
So,
2.0 1.5
P (X 2.0) = P (Z < ) = P (Z 3.19)
0.35/ 5
P (X 2.0) = (3.19) = 0.9993
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 11
Central Limit Theorem
Rule of Thumb: If n > 30, the Central Limit Theorem can be used.
Examples
Example 5.5.2 Example 5.27 in textbook. The number of major defects for a
certain model of automobile is a random variable with mean 3.2 and standard
deviation 2.4. Among 100 randomly selected cars of this model, how likely is it
that the average number of major defects exceeds 4? How likely is it that the
number of major defects of all 100 cars exceeds 20?
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)
STAT511 Spring 2014 Lecture Notes 12
Definition 16. Given a collection of n random variables X1 , X2 , , Xn and n
numerical constants a1 , a2 , , an , the rv:
n
X
Y = a1 X1 + a2 X2 + + an Xn = ai Xi
i=1
1. E(Y ) = a1 1 + a2 2 + + an n .
Example
Example 5.5.3 Example 5.30 in textbook. Three grades of gasoline are priced at
$1.20, $1.35 and $1.50 per gallon, respectively. Let X1 , X2 and X3 denote the amounts of
these grades purchased (gallons) on a particular day. Suppose Xi s are independent and
normally distributed with 1 = 1000, 2 = 500, 3 = 100, 1 = 100, 2 = 80, 3 = 50.
The total revenue of the sale of the three grades of gasoline on a particular day is
Y = 1.2X1 + 1.35X2 + 1.5X3 . Find the probability that total revenue exceeds $2500.
Purdue University Chapter5print.tex; Last Modified: February 19, 2014 (W. Sharabati)