M2S1: Probability and Statistics II

Problems

Professor David A van Dyk

Statistics Section, Imperial College London

dvandyk@imperial.ac.uk
http://www2.imperial.ac.uk/dvandyk

October 2013

1
 M2S1 Problems
(?) You may find starred problems more challenging.
() Problems marked with a () review material from M1S.

1 Introduction and Motivation

1.1 Course Administration and Syllabus
1.2 Randomized Controlled Experiments
1.2.1 Suppose you are interested in the effects of caffeine on the concentration of sleep-deprived under-
graduates, in particular on their performance on exam-like tasks. You are planning an experiment
and have identified 100 of your classmates who are willing to participate as subjects in the ex-
periment. You plan to have a test that involves reading comprehension, mathematical skills, and
analytical abilities. You plan to administer the test at 14.00 on a day when your 100 subjects
have had at most 4 hours of sleep in the last 24 hours and have had no sleep since 7.00. Describe
how you will design your experiment. Be sure to consider issues such as blinding, control and
treatment groups, control, randomization, dose, and the placebo effect. You might want to read
http://news.bbc.co.uk/1/hi/health/1142492.stm and
http://en.wikipedia.org/wiki/Effect_of_psychoactive_drugs_on_animals
for background.

2 Probability Spaces
2.1 Definitions of Probability
2.1.1 Suppose your friend has a 50p coin and is going to flip it into the air. What probability would you
assign that it will come up heads in each of the following situations:

(a) You have no information about the coin or its history of coming up head when flipped by
your friend.
(b) You friend has already flipped this coin twice, and it came up heads both times.
(c) You friend has already flipped this coin 10 times, and it came up heads each time.
(d) You friend has already flipped this coin 1000 times, and it came up heads each time.

What type of probability are you using? [Hint: No calculations are required for this question.]

2.2 Basic Probability

2.2.1 () Describe the sample spaces for each of the following experiments,

(a) A coin is tossed five times.

(b) The number of people in the queue ahead of you when you arrive in the junior common room
to purchase a decaf(!) skinny latte.
(c) Measure the life-time of a particular type of light bulb.
(d) Measure the time it takes a Piccadilly line train to go from South Kensington Station to
Heathrow Terminal 4.

2.2.2 () Find formulas for each of the following events in terms of = Pr(A), = Pr(B), and
= Pr(A B).
.........................................................................................................................................
(?) You may find starred problems more challenging. 2
() Problems marked with a () review material from M1S.
2.2 Basic Probability M2S1 Problems
(a) either A or B or both.
(b) either A or B but not both.
(c) at least one of A or B.
(d) at most one of A or B.

2.2.3 () A couple plans to have three children. There are 8 possible arrangements of girls and boys.
For example, GGB means the first two children are girls and the third child is a boy. All 8
arrangements are (approximately) equally likely.

(a) Write down all 8 arrangements of the sexes of the three children. What is the probability of
any one of these arrangements?
(b) Let X be the number of girls the couple has. What is the probability that X = 2?
(c) Starting from your work in (a), find the distribution of X. That is, what values can X take,
and what are the probabilities for each value?
(d) What named distribution does X follow? What are the mean and variance of X?

2.2.4 () If Pr(A) = 1/4 and Pr(B c ) = 1/5, can A and B be disjoint? Explain.

2.2.5 () Consider the probability space (S, B, Pr) with A, B B. Using only the Kolmogorov axioms
prove

(a) Pr(A) 1,
(b) If A B, then Pr(A) Pr(B), and
(c) Pr(A B) = Pr(A) + Pr(B) Pr(A B).

2.2.6 Let be a sample space.

(a) Show that the collection B = {, } is a sigma algebra.

(b) Let B = {all subsets of , including itself}. Show B is a sigma algebra.
(c) Show that the intersection of two sigma algebras is a sigma algebra.

2.2.7 (?) A slightly more general definition of a probability function than what we consider in class
allows a probability function to be defined on a field (rather than only on a sigma algebra). In
particular, let F be a field composed of subset of and define a probability function on F, as
a function Pr
f such that (i) Pr(A)
f 0 for all A F, (ii) Pr()
f = 1, and (iii) Pr
f is countably
f Ak ) =
additive. For a countably disjoint sequence {Ak , k = 1, 2, . . .}, we require Pr(
P f k=1
Pr(A ) only if A F.
k=1 k k=1 k
Suppose Bk F for k = 1, 2, . . . , and consider the following:

A XIOM : If B1 B2 B3 . . . and
k=1 Bk = then Pr(Bk ) 0 as k .

Now let = {1, 2, . . .} and F = {finite and cofinite subsets of }. Show that the function

0 if is finite
Pr()
f =
1 if is cofinite
does not satisfy this A XIOM. Given what we have shown about this function in class, comment
on why this result is not surprising.

2.2.8 (?) Let B be the set of countable and cocountable subsets of the real numbers, where a set is
cocountable if its complement is countable.
.........................................................................................................................................
(?) You may find starred problems more challenging. 3
() Problems marked with a () review material from M1S.
2.3 Conditional Probability and Independence M2S1 Problems
(a) Show that B is a sigma algebra. [Hint: Is a countable set of countable sets countable?]
(b) Let 
0 if is countable
P () =
1 if is cocountable.
Is this function finitely additive? Is it countable additive?

2.3 Conditional Probability and Independence

2.3.1 () Suppose Pr(A) > 0 and Pr(B) > 0, Show that

(a) if A and B are disjoint they are not independent, and

(b) if A and B are independent they are not disjoint.

2.3.2 () About one in three human twins is identical (one egg) and two in three are fraternal (two
eggs). Identical twins are always the same sex, with boys and girls being equally likely. About
one quarter of fraternal twin pairs are both boys, another quarter are both girls, and the rest of
one boy and one girl. In England and Wales about one in sixty-five births is a twin birth (http:
//www.multiplebirths.org.uk/media.asp. Let

A = {A birth in England/Wales results in twin boys}

B = {A birth in England/Wales results in fraternal twin}
C = {A birth in England/Wales results in twins}

(a) Describe the event A B C in words.

(b) Find Pr(A B C).

2.3.3 For events A and B in a sample space , under what conditions does the following hold:

Pr(A) = Pr(A|B) + Pr(A|B c )?

2.3.4 A biased coin is tossed repeatedly, with tosses mutually independent; the probability of the coin
showing Heads on any toss is p. Let Hn be the event that an even number of Heads have been
obtained after n tosses, let pn = Pr (Hn ), and define p0 = 1. By conditioning on Hn1 and using
the L AW OF T OTAL P ROBABILITY, show that, for n 1,

pn = (1 2p)pn1 + p. (1)

Find a solution to this difference equation, valid for all n 0, of the form pn = A + Bn , where
A, B and are constants to be identified. Prove that if p < 1/2, then pn > 1/2 for all n 1, and
find the limiting value of pn as n . Is this limit intuitively reasonable?

2.3.5 (?) A simple model for weather forecasting involves classifying days as either Fine or Wet, and
then assuming that the weather on a given day will be the same as the weather on the preceding
day with probability p, with 0 < p < 1. Suppose that the probability of fine weather on day
indexed 1 (say Jan 1st) is denoted by . Let n denote the probability that day indexed n is Fine.
For n = 2, 3, ..., find a difference equation for n similar to that in equation (1) in Problem 2.3.2,
and use this difference equation to find n explicitly as a function of n, p and . Find the limiting
value of n as n .

2.3.6 () Consider two coins, of which one is a normal fair coin and the other is biased so that the
probability of obtaining a Head is p > 1/2.
........................................................................................................................................
(?) You may find starred problems more challenging. 4
() Problems marked with a () review material from M1S.
 M2S1 Problems
(a) Suppose p = 1 and a coin is selected at random and tossed n times, with tosses mutually
independent. Evaluate the conditional probability that the selected coin is the normal one,
given that the first n tosses are all Heads. [Hint: You will need to use the Binomial distribu-
tion from M1S.]
(b) Now suppose 1/2 < p < 1 and that again, one of the coins is selected randomly and tossed
n times. Let E be the event that the n tosses result in k Heads and n k Tails, and let F be
the event that the coin is fair. Find Pr(F |E).

2.3.7 () A company is to introduce mandatory drug testing for its employees. The test used is very
accurate, in that it gives a correct positive test (detects drugs when they are actually present in a
blood sample) with probability 0.99, and a correct negative test (does not detect drugs when they
are not present) with probability 0.98. If an individual tests positive on the first test, a second
blood sample is tested. It is assumed that only 1 in 5000 employees actually does provide a blood
sample with drugs present.
Calculate the probability that the presence of drugs in a blood sample is detected correctly, given

(a) a positive result on the first test (before the second test is carried out),
(b) a positive result on both first and second tests.

Assume that the results of tests are conditionally independent, that is, independent given the pres-
ence or absence of drugs in the sample.

3 Univariate Random Variables and Probability Distributions

3.1 Probability Distribution, Density and Mass Functions
3.1.1 () Determine for which values of the constant c the following functions define valid probability
mass functions for a discrete random variable X, taking values on range X = {1, 2, 3, ...}. For
parts (a) and (d) the value of c will depend on . In this case, specify the range of resulting in a
valid probability mass function.

(a) fX (x) = cx , (b) fX (x) = c/(x2x ),

(c) fX (x) = c/(x2 ), (d) fX (x) = cx /x!.

In each case, calculate Pr(X > 1).

3.1.2 () Suppose n identical fair coins are tossed. Those that show Heads are tossed again, and the
number of Heads obtained on the second set of tosses defines a discrete random variable X.
Assuming that all tosses are independent, find the range and probability mass function of X.

3.1.3 () A continuous random variable X has pdf given by

fX (x) = c(1 x)x2 , for 0 < x < 1,

and zero otherwise. Find the value of c, the cdf of X, FX , and Pr(X > 1/2).

3.1.4 () A function f is defined by

f (x) = k/xk+1 , for x > 1,

and zero otherwise. For what values of k is f a valid pdf? Find the cdf of X.
........................................................................................................................................
(?) You may find starred problems more challenging. 5
() Problems marked with a () review material from M1S.
3.2 Transformations of Univariate Random Variables M2S1 Problems
3.1.5 () A continuous random variable X has pdf given by
(
x, for 0 < x < 1,
fX (x) =
2 x, for 1 x < 2,

and zero otherwise. Plot fX , find the cdf FX , and plot FX .

3.1.6 () Show that the function, FX , defined for x R by

n o
FX (x) = c exp ex

is a valid cdf for a continuous random variable X for a specific choice of constant c, where > 0.
Find the pdf fX associated with this cdf.

3.1.7 Evaluate Z
2
e4x dx.
0
[Hint: Relate the integrand to a well-known pdf.]

3.1.8 A point is to be selected randomly from an integer lattice restricted to the triangle with corners at
(1, 1), (n, 1) and (n, n) for positive integer n. If all points are equally likely to be selected, find
the probability mass functions for the two discrete random variables X and Y corresponding to
the x and y coordinates of the selected point respectively.

3.1.9 Consider two random variables, X with cdf FX and Y with cdf FY . We say that Y is stochastically
greater than X if FY (u) FX (u) for all u and FY (u) < FX (u) for some u. If Y is stochastically
greater than X, prove that

Pr(Y > u) Pr(X > u) for every u

and
Pr(Y > u) > Pr(X > u) for some u.
Qualitatively compare the distributions of X and Y .

3.2 Transformations of Univariate Random Variables

3.2.1 () Suppose that X is a continuous random variable with density function given by

fX (x) = 4x3 , for 0 < x < 1,

and zero otherwise. Find the density functions of the following random variables:

(a) Y = X 4 , (b) W = eX , (c) Z = log X, (d) U = (X 0.5)2 .

3.2.2 Again suppose that X is a continuous random variable with density function given by

fX (x) = 4x3 , for 0 < x < 1,

and zero otherwise. Find the monotonic decreasing function H such that the random variable V ,
defined by V = H(X), has a density function that is constant on the interval (0, 1), and zero
otherwise.

.........................................................................................................................................
(?) You may find starred problems more challenging. 6
() Problems marked with a () review material from M1S.
3.3 Expected Values M2S1 Problems
3.2.3 The measured radius of a circle, R, is a continuous random variable with density function given
by
fR (r) = 6r(1 r), for 0 < r < 1,
and zero otherwise. Find the density functions of (a) the circumference and (b) the area of the
circle.

3.2.4 Suppose that X is a continuous random variable with density function given by
(+1)
x

fX (x) = 1+ , for x > 0,

and zero elsewhere, with and non-negative parameters.

(a) Find the density function and cdf of the random variable defined by Y = log X.
(b) Find the density function of the random variable defined by Z = + Y .

3.3 Expected Values

3.3.1 () onsider the function fX (x) = cg(x) for some constant c > 0, with g defined by

|x|
g(x) = , for x R.
(1 + x2 )2

Show that fX (x) is a valid pdf for a continuous random variable X with range X = R, and find
the cdf, FX , and the expected value of X, E(X), associated with this pdf.

3.3.2 Suppose that X is a continuous random variable with support R. Its pdf is given by

fX (x) = 2 x exp {x} , for x 0,

and is zero otherwise, with parameter > 0.

(a) Find the cdf of X.

(b) Show that, for any positive value m, Pr(X m) = (1 + m) exp {m} .
(c) Find E(X).
(d) Now consider the pdf
gX (x) = kx2 exp{x}, for x 0 (2)
and zero elsewhere. Find the value of k for which (2) is a valid pdf.

3.3.3 The solid line in the left panel of Figure 1 is an unnormalized density function, fX (x). It does
not integrate to one and its integral is unknown. It is superimposed on a standard normal pdf,
(x), (dashed line). The ratio fX (x)/(x) is plotted in the right panel. Suppose you obtain a
large random sample from the standard normal distribution, (X1 , X2 , . . . , Xn ). How might you
use this sample to approximate the expectation under fX (x), the properly normalized version of
fX (x)? Based on the plot in Figure 1, how well do you expect your method to work? Why?

.........................................................................................................................................
(?) You may find starred problems more challenging. 7
() Problems marked with a () review material from M1S.
3.4 Higher Moments M2S1 Problems

0.4

1.2
0.3

fX(x) (x)
1.0
fX(x)
0.2

0.8
0.1

0.6
0.0

3 2 1 0 1 2 3 3 2 1 0 1 2 3
x x

Figure 1: Plots for Problem 3.3.3.

3.4 Higher Moments

3.4.1 Suppose that X is a continuous random variable with pdf
1 x/
fX (x) = e , for x > 0

p
and zero elsewhere, with a positive parameter. Let Y = 2X/. Find the pdf, mean, and
variance of Y .
3.4.2 A continuous random variable X has cdf given by

FX (x) = c(x x ), for 0 x 1,

FX (x) = 0, for x < 0 and FX (x) = 1, for x > 1, for constants 1 < . Find the value of
constant c, and evaluate the rth moment of X.
3.4.3 Let X be a random variable with k th non-central moment, k = E(X k ), and k th central moment,
k = E[(X 1 )k ]. In addition to the mean and variance, two quantities sometimes used to
summarize a distribution are the
3 4
skewness : 3 = 3/2
and kurtosis : 4 = 2 .
(2 ) 2

The skewness measures asymmetry in the pdf, while the kurtosis measures peakedness. A pdf
is said to be symmetric around a point m if fX (m ) = fX (m + ) for every positive .
(a) Show that the skewness of a symmetric distribution is zero.
(b) Show 3 = 3 32 1 + 213 .
(c) Suppose (
0 with probability 1 p
X=
with probability p,
where > 0. Find and plot the skewness of X as a function of p and .
........................................................................................................................................
(?) You may find starred problems more challenging. 8
() Problems marked with a () review material from M1S.
 M2S1 Problems
(d) Calculate the skewness of fX (x) = ex , for x 0 and 0 elsewhere. This is a right skewed
distribution.
(e) Calculate the kurtosis for each of the following pdfs and comment on the peakedness of
each.
1
for x R
2
(i) fX (x) = ex /2 ,
2
1
(ii) fX (x) = , for 1 < x < 1
2
1
(iii) fX (x) = e|x| , for x R
2

3.4.4 (?) Let X be a continuous random variable with range X = R+ , pdf fX and cdf FX .

(a) Show that Z

E(X) = [1 FX (x)] dx.
0

(b) Show also that for integer r 1,

Z
r
E(X ) = rxr1 [1 FX (x)] dx.
0

(c) Find a similar expression for random variables for which X = R.

4 Univariate Families of Distributions

4.1 Standard Univariate Parametric Families
4.1.1 ()

(a) At a certain London university, the average overall grade of students in their first year is 65
with a standard deviation of 15. Under a normal model, what proportion of students have
grades over 80? Under 40? Find the sixtieth percentile of the overall grades of first year
students. If there are 250 first year students, find the mean and standard deviation of the
number who score above 80.
(b) About 75% of 20 year old women weigh between 103.5 and 148.5 lb. Using a normal model,
and assuming that 103.5 and 148.5 are equal distant from the mean, , calculate .

4.1.2 One reason cited for the mental deterioration so often seen in the very elderly is the reduction
in cerebral blood flow that accompanies the aging process. Addressing itself to this notion, a
study was done (Ball and Taylor, 1969) to see whether cyclandelate, a vasodilator, might be able
to stimulate the cerebral circulation and thereby slow the rate of mental deterioration. Blood
circulation time can be measured using a radioactive tracer. Let X and Y be the mean blood
circulation time before treatment and after treatment respectively, for a randomly selected elderly
patient.

(a) Let 
0 if Y < X
D=
1 if Y > X.
What is the distribution of D?

.........................................................................................................................................
(?) You may find starred problems more challenging. 9
() Problems marked with a () review material from M1S.
4.2 Classes of Parametric Families M2S1 Problems
(b) Consider the skeptical hypothesis that cyclandelate has no effect on mean circulation time
and any differences that are observed before and after treatment are due to chance. What is
the distribution of D under this hypothesis?
Pn
(c) Now suppose we select a random sample of n patients, and let = i=1 Di , what is the
distribution of under the skeptical hypothesis?
(d) The drug was given to eleven subjects and blood flow was measured before and after treat-
ment as described above. The data appear below. What is the observed value, , of the
random variable ? How likely is it that we would see a value as extreme or more extreme
under the skeptical hypothesis? What do you conclude about the skeptical hypothesis?

C EREBRAL C IRCULATION E XPERIMENT

Mean Circulation Time (seconds)
Before, xi After, yi
15 13
12 8
12 12.5
14 12
13 12
13 12.5
13 12.5
12 14
12.5 12
12 11
12.5 10

4.1.3 The drug enforcement team in a large city ceased a stash of 496 small packets containing what
appeared to be illegal narcotics. They were packaged for resale on the street. Four of the packets
were randomly sampled, tested, and found to contain prohibited substances. Undercover police
officers took two more of the (untested) packets and sold them to a defendant later accused of
purchasing illegal drugs. Unfortunately, these last two packets were lost before they could be
tested for narcotics. [This question is based on actual events as described in Shuster (1991) and
reported in Casella and Berger (2002).]

(a) Let N be the number of the original 496 packets that contained prohibited substances and
let M = 496 N be the number that did not. Compute the probability that the first four
randomly selected packets contained narcotics and the next two randomly selected packets
did not. (You should report your answer as a function of N and M .)
(b) Maximize the probability that you found in part (a) as a function of N and M = 496 N .
This is the defendants maximum probability of innocence.

4.1.4 Show that the binomial distribution converges to the Poisson distribution as n with = pn
held fixed. That is,
!
n x e x
lim p (1 p)nx = ,
n x x!
where p = /n.

4.2 Classes of Parametric Families

4.2.1 Consider the Poisson distribution with expectation .

........................................................................................................................................
(?) You may find starred problems more challenging. 10
() Problems marked with a () review material from M1S.
 M2S1 Problems
(a) Show that the Poisson distribution belongs to the exponential family. What is the canonical
parameterization?
nP o
J
(b) Now suppose = exp j=1 xj j , where (x1 , . . . , xJ ) are known predictor variables,
and (1 , . . . , J ) are unknown parameters. (Here replaces as the unknown parameter.)
Show that this new distribution is also a member of the exponential family.
4.2.2 Consider the probability density function fX (x) = ex , for x > 0.
(a) Construct a location-scale family fX (x|, ) from fX (x), where is the location parameter
and is the scale parameter.
(b) Derive the non-central moments, n = E(Y n ), where Y follows the location-scale family.

5 Multivariate Random Variables and Probability Distributions

5.1 Multivariate Random Variables
5.1.1 Suppose that X and Y are discrete random variables with joint mass function given by
x+y
fX,Y (x, y) = c 2x!y! , for x, y = 0, 1, 2, . . . ,
and zero otherwise, for some constant c.

(a) Find the value of c, and the marginal mass functions of X and Y .
(b) Prove that X and Y are independent random variables, that is,
fX,Y (x, y) = fX (x)fY (y), for all x, y = 0, 1, . . . .
5.1.2 Continuous random variables X and Y have joint cdf FX,Y defined by
1 1
 
x

FX,Y (x, y) = 1 e + tan1 y , for x > 0 and < y < ,
2
with
FX,Y (x, y) = 0, for x 0.
Find the joint pdf, fX,Y . Are X and Y independent ? Justify your answer.
5.1.3 Suppose that X and Y are continuous random variables with joint pdf given by
fX,Y (x, y) = cx(1 y), for 0 < x < 1 and 0 < y < 1,
and zero otherwise, for some constant c.

(a) Are X and Y independent random variables?

(b) Find the value of c.
(c) Find Pr(X < Y ).

5.1.4 Suppose that the joint pdf of X and Y is given by

fX,Y (x, y) = 24xy, for x > 0, y > 0, and x + y < 1,
and zero otherwise. Find (a) the marginal pdf of X, fX , (b) the marginal pdf of Y , fY , (c) the
conditional pdf of X given Y = y, fX|Y , (d) the conditional pdf of Y given X = x, fY |X , (e) the
expected value of X, and (f) the expected value of Y .
[Hint: Sketch the region on which the joint density is non-zero; remember that the integrand is
only non-zero for some part of the integral range.]
........................................................................................................................................
(?) You may find starred problems more challenging. 11
() Problems marked with a () review material from M1S.
5.2 Multivariate Transformations M2S1 Problems
5.1.5 Suppose that X and Y are continuous random variables with joint pdf given by
1 1
fX,Y (x, y) = 2
, for 1 x < and y x,
2x y x
and zero otherwise.

(a) Find the marginal pdf of X.

(b) Find the marginal pdf of Y .
(c) Find the conditional pdf of X given Y = y.
(d) Find the conditional pdf of Y given X = x.
(e) Find the expectation of Y , E(Y ).

5.1.6 Suppose X is a random variable with pdf proportional to the normal pdf (mean and variance
2 ) for positive x and zero elsewhere,
(x)2
n o
k
22 exp 22
, for x 0
fX (x) =

0, elsewhere .

Derive the mean and variance of X.

5.1.7 A critical component in an experimental rocket can withstand temperatures up to t0 in degrees

Centigrade. If the maximum temperature, T , of this component exceeds t0 , the rocket will fail.
Preliminary tests indicate that there is some variability in the maximum temperature that the com-
ponent is likely to reach when the rocket is launched; the pdf of this temperature is given by fT (t).
Engineers are anxious about an upcoming test launch because, although t
R
0
f T (t)dt is near zero,
it is greater than zero. There are sensors in the rocket that will record T , but it will take some time
to recover the rocket and analyze the data from the sensors. (This, of course, is assuming that the
rocket does not fail.) Suppose that the test launch goes smoothly and the rocket does not fail.

(a) Carefully derive the pdf of the maximum temperate of the critical component after the en-
gineers observe the successful launch of the rocket, but before they are able to analyze the
sensor data.
(b) Verify that the pdf you gave in part (a) is a valid pdf.

5.2 Multivariate Transformations

iid
5.2.1 Suppose Xi Gamma(i , ) for i = 1, . . . , n.

(a) Use the convolution theorem to show that Y = X1 + X2 Gamma(1 + 2 , ).

Pn Pn
(b) Prove that Zn = i=1 Xi Gamma ( i=1 i , ).

5.2.2 (?) Suppose that X and Y have joint pdf that is constant on the range X(2) (0, 1) (0, 1), and
zero otherwise. Find the marginal pdf of the random variables U = X/Y and V = log(XY ),
stating clearly the range of the transformed random variable in each case.
[Hint: For U , you might consider first the joint pdf of (U, X), then obtain the marginal pdf of U .
For V , consider the joint pdf of (V, log X), then obtain the marginal pdf of V . Compare the
ease of these calculations with those required by the joint transformation from (X, Y ) to (U, V ).]

........................................................................................................................................
(?) You may find starred problems more challenging. 12
() Problems marked with a () review material from M1S.
5.2 Multivariate Transformations M2S1 Problems
5.2.3 (?) Suppose that continuous random variables X1 , X2 , X3 are independent, and have marginal
pdfs specified by
fXi (xi ) = ci xii exi , xi > 0,
for i = 1, 2, 3, where c1 , c2 and c3 are normalizing constants. Find the joint pdf of the random
variables Y1 , Y2 , Y3 defined by
Y1 = X1 /(X1 + X2 + X3 ), Y2 = X2 /(X1 + X2 + X3 ), Y3 = X1 + X2 + X3 ,
and evaluate the (marginal) expectation of Y1 .
5.2.4 Suppose that X and Y are continuous random variables with pdf given by
1 1
 
exp x2 + y 2 , for x, y R.

fX,Y (x, y) =
2 2
(a) Let the random variable U be defined by U = X/Y . Find the pdf of U .
(b) Suppose now that S 2 is independent of X and Y . (The pdf of S is given by
fS (s) = c()s/21 es/2 , for s > 0,
where is a positive integer and c() is a normalizing constant depending on .) Find the
pdf of random variable T defined by
X
T =p .
S/
This is the pdf of a t random variable with degrees of freedom.

5.2.5 Suppose (X1 , . . . , Xn ) is a collection of independent and identically distributed random variables
taking values on X with pmf/pdf fX and cdf FX . Let Yn and Zn correspond to the maximum and
minimum order statistics derived from (X1 , . . . , Xn ), that is
Yn = max {X1 , . . . , Xn } , Zn = min {X1 , . . . , Xn } .
(a) Show that the cdfs of Yn and Zn are given by
FYn (y) = {FX (y)}n , FZn (z) = 1 {1 FX (z)}n .
(b) Suppose X1 , . . . , Xn Unif(0, 1), that is
FX (x) = x, for 0 x 1.
Find the cdfs of Yn and Zn .
(c) Suppose X1 , . . . , Xn have cdf
FX (x) = 1 x1 , for x 1.
Find the cdfs of Zn and Un = Znn .
(d) Suppose X1 , . . . , Xn have cdf
1
, for x R. FX (x) =
1 + ex
Find the cdfs of Yn and Un = Yn log n.
(e) Suppose X1 , . . . , Xn have cdf
1
FX (x) = 1 , for x > 0.
1 + x
Find the cdfs of Yn , Zn , Un = Yn /n, and Vn = nZn .
........................................................................................................................................
(?) You may find starred problems more challenging. 13
() Problems marked with a () review material from M1S.
5.3 Covariance and Correlation M2S1 Problems
5.3 Covariance and Correlation
5.3.1 Suppose X and Y are two random variables each with finite mean and variance. Prove 1
XY 1 by using the fact that
X Y X Y
   
Var + and Var
X Y X Y
are both positive quantities.

5.3.2 Suppose that X and Y have joint pdf given by

fX,Y (x, y) = cxy(1 x y), for 0 < x < 1, 0 < y < 1, and 0 < x + y < 1,

for some constant c > 0. Find the covariance of X and Y .

5.4 Hierarchical and Mixture Models

5.4.1 Suppose X|Y Exponential(Y ) and Y Gamma(, ), using the parameterization on the
formula sheet.

(a) Find the mean and variance of X.

(b) Find the marginal distribution of X.

5.4.2 The number of daughters of an organism is a discrete random variable with mean and variance
2 . Each of its daughters reproduces in the same manner. Find the expectation and variance of
the number of granddaughters.

5.4.3 Suppose that the joint pdf of random variables X and Y is specified via the conditional density
fX|Y and the marginal density fY as
( )
yx2
r
y
fX|Y (x|y) = exp , for x R; fY (y) = c()y /21 ey/2 , for y > 0,
2 2

where is a positive integer. Find the marginal pdf of X.

6 Multivariate Families of Distributions

6.1 Multinomial and Dirichlet Distributions
6.1.1 Suppose N Poisson() and X|N Multinomial(N, p), where N is univariate and X =
(X1 , . . . , Xk )> is a (k 1) random variable and p = (p1 , . . . , pk )> is a (k 1) probability
vector.

(a) Write the joint pmf of N and X.

ind
(b) (?) Rearrange the terms in the joint pmf and its support to show that Xi Poisson(pi )
and N = ki=1 Xi .
P

........................................................................................................................................
(?) You may find starred problems more challenging. 14
() Problems marked with a () review material from M1S.
6.2 Multivariate Normal Distribution M2S1 Problems
6.2 Multivariate Normal Distribution
6.2.1 (?) The Bivariate Normal Distribution: Suppose that X1 and X2 are independent and identically
distributed N(0, 1) random variables. Let random variables Y1 and Y2 be defined by
q
Y1 = 1 + 1 1 2 X1 + 1 X2 and Y2 = 2 + 2 X2 ,

or, equivalently, p !
1 1 2 1
    
Y1 1 X1
= + ,
Y2 2 0 2 X2
for positive constants 1 and 2 , and || < 1.

(a) Find the joint pdf of (Y1 , Y2 ).

(b) Show that, marginally for i = 1, 2, Yi N i , i2 , and that conditionally

h
i
1
Y1 |Y2 = y2 N 1 + 2 (y2 2 ) , 12 1 2
h
i
2
Y2 |Y1 = y1 N 2 + 1 (y1 1 ) , 22 1 2 .

(c) Find the correlation between Y1 and Y2 .

6.2.2 Suppose
0.5
      
X1 2 1
N2 = , = .
X2 5 0.5 4
Compute Pr(X1 > 0) and Pr(X2 < 6).
[Hint: You may use the result of Problem 6.2.1.]

6.2.3 The joint pdf of the random variables X1 and X2 is

( !)
x21 x1 x2 2x22
fX1 ,X2 (x1 , x2 ) = k exp + , for < x1 , x2 < .
6 3 3

Find E(X1 ), E(X2 ), Var(X1 ), Var(X2 ), Cov(X1 , X2 ) and k.

[Hint: You may use the result of Problem 6.2.1.]

6.2.4 (?) [Warning: If you have an aversion to vector notation, you may find this question challenging!]
Suppose Y and X = (X1 , X2 )> jointly follow a trivariate normal distribution. Here Y is a
univariate random variable and Z = (Y, X1 , X2 )> is a (3 1) trivariate normal random vector
with mean
   1
Y 1 mY Y MY X
= and variance-covariance matrix M = ,
X MY>X MXX

where Y is the univariate mean of Y , X is the (2 1) mean vector of X, is the (3 1) mean

vector of both X and Y , mY Y is the first diagonal element of M , MXX is the lower-right (22)
submatrix of M , and MY X is the remaining off-diagonal (1 2) submartix of M . (Note that
we parameterize the multivariate normal in terms of the inverse of its variance-covariance matrix.
This will significantly simplify calculations!)

(a) Derive the conditional distribution of Y given both X1 and X2 . [Hint: Use vector/matrix
notation.]
........................................................................................................................................
(?) You may find starred problems more challenging. 15
() Problems marked with a () review material from M1S.
6.3 Connections between the Distributions M2S1 Problems
>
(b) Now suppose Y and X = (X1 , . . . , Xn ) jointly follow a multivariate normal distribution.
Here Y remains a univariate random variable and Z = (Y, X1 , . . . , Xn )> is an [(n + 1) 1]
multivariate normal random vector. Use the same notation for the mean and the inverse
of the variance-covariance matrix, but with appropriately adjusted dimensions. Derive the
conditional distribution of Y given X1 , . . . , Xn . [Hint: If you used vector/matrix notation
in part (a), this problem will be very easy. If you did not, it will be very hard!]
(c) Set n = 1 and check that your answer is the same as the conditional distribution for the
bivariate normal derived in lecture and in Problem 6.2.1.

6.3 Connections between the Distributions

6.3.1 Suppose that U1 and U2 are independent and identically distributed Unif(0, 1) random variables.
Let random variables Z1 and Z2 be defined by
q
Z1 = 2 log(U1 ) cos (2U2 ) ,
q
Z2 = 2 log(U1 ) sin (2U2 ) .

Find the joint pdf of (Z1 , Z2 ).

6.3.2 Suppose that U is a Unif(0, 1) random variable. Find the distribution of

X = log U.

6.3.3 Suppose that an unlimited sequence of Unif(0, 1) random variables is available. Using the results
of Problems 6.3.1 and 6.3.2, and results discussed earlier this term, describe how to generate:

(a) a Gamma(k, ) random variable, for integer k 0;

(b) a realization of a Poisson process with rate ;
 
1
(c) a 2 Gamma 2, 2 random variable, where is a positive, integer parameter;
(d) a tn random variable, where n is a positive integer parameter.

7 Sampling Distributions and Statistical Inference

7.1 Background
7.2 Statistics and Their Sampling Distributions
7.2.1 Suppose that (X1 , . . . , Xn ) is a random sample from a Poisson() distribution. Define the statis-
tics
n n
1X 1 X
T1 = X = Xi , and T2 = S 2 = (Xi X)2 .
n i=1 n 1 i=1
Show that
E(T1 ) = E(T2 ) = .

7.2.2 Suppose that (X1 , . . . , Xn ) is a random sample from the probability distribution with pdf
1
fX (x; ) = ex/ , for x > 0.

........................................................................................................................................
(?) You may find starred problems more challenging. 16
() Problems marked with a () review material from M1S.
7.3 The Method of Moments M2S1 Problems
(a) Show that the sample mean X is an unbiased estimator of .
(b) Set Y1 = min {X1 , . . . , Xn } and show that Z = nY1 is also unbiased for .

7.2.3 Suppose that (X1 , . . . , Xn ) is a random sample from the uniform distribution on ( 1, + 1).

(a) Show that the sample mean X is an unbiased estimator of .

(b) Let Y1 and Yn be the smallest and largest order statistics derived from (X1 , . . . , Xn ). Show
also that random variable M = (Y1 + Yn ) /2 is an unbiased estimator of .

7.3 The Method of Moments

7.3.1 Method of moments.

(a) Suppose (X1 , . . . , Xn ) is a random sample from a gamma distribution, having pdf
1
fX (x) = x1 exp{x/}, for x > 0,
()

where , > 0. Find the method of moments estimators of and . Can the corresponding
estimates ever be outside the parameter space?
(b) Suppose (X1 , . . . , Xn ) is a random sample from a beta distribution, having pdf
1
fX (x) = x1 (1 x)1 , for 0 < x < 1,
B(, )

where , > 0. Find the method of moments estimators of and . Can the corresponding
estimates ever be outside the parameter space?

7.4 Maximum Likelihood Estimation

7.4.1 [Problem 4.2.2 continued] Consider the probability density function fX (x) = ex for x > 0.
In Problem 4.2.2 (a) you derived a location-scale family fX (x|, ) from fX (x), where is the
location parameter and is the scale parameter. Now suppose = 1. Report the loglikelihood
function for and compute its maximum likelihood estimator, .

7.4.2 [Problem 6.1.1 continued] Suppose N Poisson() and X|N Multinomial(N, p), where N
is univariate and X = (X1 , . . . , Xk )> is a (k 1) random variable and p = (p1 , . . . , pk )> is a
(k 1) probability vector. In Problem 6.1.1, you showed that
k
ind X
Xi Poisson(pi ) and N = Xi . (3)
i=1

(a) Let = p. Note that = (1 , . . . , k ) is a (k 1) vector. Using (3), find the maximum
likelihood estimator, of .
(b) Using derived in part (a) derive formulas for and p that satisfy = p, such that p is
a probability vector. You do not need to show it, but maximum likelihood estimators are
invariant to transformations, so that and p are the maximum likelihood estimators of and
p, respectively.

7.4.3 Suppose that (X1 , . . . , Xn ) is a random sample from a Poisson() distribution.

(a) Find the maximum likelihood estimator of and show that this estimator is unbiased.
........................................................................................................................................
(?) You may find starred problems more challenging. 17
() Problems marked with a () review material from M1S.
7.5 Random Intervals and Confidence Intervals M2S1 Problems
(b) Find the maximum likelihood estimator of () = e = Pr(X = 0).

7.4.4 Find the maximum likelihood estimators of the unknown parameters in the following probability
densities on the basis of a random sample of size n.

(a) fX (x; ) = x1 , for 0 < x < 1 and > 0.

(b) fX (x; ) = ( + 1)x2 , for 1 < x and > 0.
(c) fX (x; ) = 2 x exp {x} , for 0 < x and > 0.
(d) fX (x; ) = 22 x3 , for x and > 0.
(e) fX (x; 1 , 2 ) = 1 21 x1 1 , for 2 x and 1 , 2 > 0.

7.5 Random Intervals and Confidence Intervals

7.5.1 Suppose you observe a single observation from a normal distribution with unit variance and un-
known mean, . Specifically, X N(, 1).

(a) Report the loglikelihood function for and compute its maximum likelihood estimator, .
(b) What is the distribution of ?
(c) Derive an interval I() that has a 95% chance of containing . For definiteness, choose the
shortest possible interval with this property. Make a plot of your interval, with plotted on
the horizontal axis, and the lower and upper bounds of I() plotted on the vertical axis.
n o
(d) Now consider the interval J () = : I() . Identify this interval on your plot from
your answer to part (c). Give formulas for the lower and upper bounds of J (). Notice
that J can be computed from data, whereas I cannot. (I depends on the unknown mean, ,
while J depends only on the maximum likelihood estimator.)
(e) Show that h i
Pr J () = 95%.
[Hint: What is the random quantity in this expression.] An interval with this property is
called a 95% confidence interval.

7.5.2 Suppose you observe a binomial random variable, X Bin(n, p), where n is known, but p is not.

(a) Report the loglikelihood function for p and compute its maximum likelihood estimator, p.
(b) What is the distribution of p?
(c) Suppose n = 10 and for p on the grid of values (0, 0.1, 0.2, 0.3, . . . , 1.0), derive an interval
I(p) that has at least a 95% chance of containing p. For definiteness, choose the shortest
possible interval with this property. Make a plot of your interval, interpolating linearly
between grid points, with p plotted on the horizontal axis, and the lower and upper bounds
of I(p) plotted on the vertical axis.
n o
(d) Now consider the interval J (p) = p : p I(p) . Identify this interval on your plot from
your answer to part (c). Compute J (p) for each possible value of p.
(e) Show that h i
Pr p J (p) 95%,

at least for p (0, 0.1, 0.2, 0.3, . . . , 1.0).

(f) Qualitatively, how will the intervals change as n increases?

........................................................................................................................................
(?) You may find starred problems more challenging. 18
() Problems marked with a () review material from M1S.
7.6 Bayesian Statistical Inference M2S1 Problems
7.5.3 Suppose that (X1 , . . . , Xn ) is a random sample from the probability distribution with pdf

fX (x; ) = ex , for x > 0.

(a) Find the maximum likelihood estimator of and show that it is biased as an estimator of ,
but that some multiple of it is not.
(b) Show that 2 ni=1 Xi is a pivotal quantity. Describe briefly how to use this to construct a
P

100(1 )% confidence interval for , (0, 1).

7.5.4 Let (X1 , . . . , Xn ) be a random sample from the uniform distribution on (0, ).

(a) Find the maximum likelihood estimator, of .

(b) By considering the distribution of /, show that for (0, 1), a 100(1 )% confidence
interval for based on is given by (, /1/n ).

7.5.5 Suppose X1 N(1 , 1) and X2 N(2 , 1), with X1 and X2 independent and 1 and 2 both
unknown. Show that both the square S and circle C given by

S = {(1 , 2 ) : |X1 1 | 2.236, |X2 2 | 2.236} and

C = {(1 , 2 ) : (X1 1 )2 + (X2 2 )2 5.991}

are 95% confidence sets for (1 , 2 ). What is a sensible criterion for choosing between S and C?

7.5.6 The following data is sampled from a N(, 2 ) distribution, where both and 2 are unknown:
6.82, 6.07, 3.74, 6.87, 5.92. For this data xi = 29.42 and x2i = 179.588.
P P

(a) Find a 95% confidence interval for and show that its width is about 3.16.
(b) Suppose it becomes known that the true value of 2 is 1. Show that a 95% confidence
interval for now has width about 1.75. The width of the confidence interval is narrower
when the true value of 2 is known. Will this always happen?
(c) (?) Consider the event that the 95% confidence interval for is narrower when 2 is known
than when it is unknown, with both intervals are computed from the same random sample
(X1 , . . . , Xn ) from N(, 2 ). Show that for n = 5 this event has probability a bit less than
0.75. [Hint: you will need to refer to tables of the quantiles of the 24 distribution.]

7.6 Bayesian Statistical Inference

7.6.1 Suppose Y | = Poisson() and we wish to estimate using Bayesian methods and specify a
gamma prior distribution, Gamma(r, ). Use the parameterization of the gamma distribution
on the formula sheet and assume that r is a positive integer.

(a) Derive the posterior distribution of given Y . What named distribution is this?
(b) What are the posterior mean and variance of ?
(c) Find the marginal distribution of Y . What named distribution is this?
(d) Compute the maximum likelihood estimator, of . How does the maximum likelihood
estimate compare with the posterior mean?
[Hint: Ignore the prior distribution when computing the maximum likelihood estimate/estimator!]

7.6.2 Suppose Y | Binomial(n, ) and show:

(a) If Unif(0, 1), then Var(|Y ) < Var().

........................................................................................................................................
(?) You may find starred problems more challenging. 19
() Problems marked with a () review material from M1S.
 M2S1 Problems
(b) If Beta(, ), Var(|Y ) may be larger than Var().

7.6.3 Suppose you are given the choice between two envelopes, one containing pounds and the other
with 2 pounds. The envelopes are sealed and shuffled so that you do not know which one contains
more money. One of the envelopes is opened and found to have x pounds. You can either take the
x pounds or take the other envelope and the money that it contains. You are not, however, allowed
to open the second envelope before you make your decision.

(a) Suppose that your subjective pdf for is uniform on the interval (0, M ) for some positive
M . What is your optimal strategy given x.
(b) Now suppose that your subjective pmf for is f () = 1/10 for = 1, 2, . . . , 10. What is
your optimal strategy given x.
(c) Finally, suppose that your subjective pdf for is f () = 1 e/ for > 0 and 0 elsewhere,
where is some positive number. Show how the optimal strategy depends on x. [Hint: Let
W be the expected value of the money in the sealed envelope and let = W/x. When is
> 1?]

8 Convergence Concepts
8.1 Convergence in Distribution and the Central Limit Theorem
8.1.1 Suppose that random variable X has mgf, MX (t) given by
1 2 5
MX (t) = et + e2t + e3t .
8 8 8
Find the probability distribution, expectation, and variance of X.
[Hint: Consider MX and its definition.]

8.1.2 Suppose that X is a continuous random variable with pdf

fX (x) = exp {(x + 2)} , for 2 < x < .

Find the mgf of X, and hence find the expectation and variance of X.

8.1.3 Suppose Z N(0, 1).

(a) Find the mgf of Z, and also the pdf and the mgf of the random variable X, where

1
X =+ Z,

for parameters and > 0.
(b) Find the expectation of X, and the expectation of the function g(X), where g(x) = ex . Use
both the definition of the expectation directly and the mgf and compare the complexity of
your calculations.
(c) Suppose now Y is the random variable defined in terms of X by Y = eX . Find the pdf of
Y , and show that the expectation of Y is
1
 
exp + 2 .
2

........................................................................................................................................
(?) You may find starred problems more challenging. 20
() Problems marked with a () review material from M1S.
8.1 Convergence in Distribution and the Central Limit Theorem M2S1 Problems
2
(d) Finally, let random variable T be defined by T = Z . Find the pdf and mgf of T .
8.1.4 Suppose that X is a random variable with pmf/pdf fX and mgf MX . The cumulant generating
function of X, KX , is defined by KX (t) = log [MX (t)]. Prove that
d d2
{KX (t)}t=0 = E(X), {KX (t)}t=0 = Var(X).
dt dt2
8.1.5 Using the C ENTRAL L IMIT T HEOREM, construct Normal approximations to each of the following
random variables,

(a) a Binomial distribution, X Binomial(n, );

(b) a Poisson distribution, X Poisson();
(c) a Negative Binomial distribution, X Negative Binomial(n, ).

8.1.6 Let Sn2 denote the sample variance of a random sample of size n from N(, 2 ), so that
(n 1)Sn2
2n1 . Vn =
2
Show, using the C ENTRAL L IMIT T HEOREM that

n 1(Sn2 2 ) D
Z N(0, 1),
2 2
 4

2
so that, for large n, Sn2 is approximately distributed as N 2 , n1 .

8.1.7 [Problem 5.2.5 continued] In Problem 5.2.5, you derived the cdfs of a number of random variables
involving the minimum or maximum of a random sample. In this problem we will derive the
limiting distribution of these same random variables.
Suppose (X1 , . . . , Xn ) is a collection of independent and identically distributed random variables
taking values on X with pmf/pdf fX and cdf FX , let Yn and Zn correspond to the maximum and
minimum order statistics derived from X1 , . . . , Xn .

(a) Suppose X1 , . . . , Xn Unif(0, 1), that is

FX (x) = x, for 0 x 1.
Find the limiting distributions of Yn and Zn as n .
(b) Suppose X1 , . . . , Xn have cdf
FX (x) = 1 x1 , for x 1.
Find the limiting distributions of Zn and Un = Znn as n .
(c) Suppose X1 , . . . , Xn have cdf
1
FX (x) = , for x R.
1 + ex
Find the limiting distributions of Yn and Un = Yn log n, as n .
(d) Suppose X1 , . . . , Xn have cdf
1
FX (x) = 1 , for x > 0.
1 + x
Let Un = Yn /n and Vn = nZn . Find the limiting distributions of Yn , Zn , Un , and Vn as
n .
........................................................................................................................................
(?) You may find starred problems more challenging. 21
() Problems marked with a () review material from M1S.
8.2 Convergence in Probability, the Law of Large Numbers, and Inequalities M2S1 Problems
8.2 Convergence in Probability, the Law of Large Numbers, and Inequalities
8.2.1 Convergence in Probability. Suppose X1 , . . . , Xn Poisson(). Let
n
1 X
X = Xi .
n i=1

p
(a) Show that X as n .
p
(b) (?) Suppose Tn = eX , show that Tn e .

8.2.2 Suppose S 2 is computed from

a random sample, (X1 , . . . , Xn ), from a distribution with finite
variance, . Letting S = S 2 , show E(S) and if 2 > 0, then E(S) < .
2

........................................................................................................................................
(?) You may find starred problems more challenging. 22
() Problems marked with a () review material from M1S.