Anda di halaman 1dari 3

STAT211 Random Processes

Assignment 3, due on May 20th, 2016 at 2pm

Q1. Suppose a piece of equipment can be in one of three states:
1 - in use;
2 - under repair; and
3 - idle awaiting more work.
We observe this equipment only after it changes state.
Let Xn be a random variables for the state the equipment has changed
into at timestep n.
Assume that if the equipment is under repair, then the next change
will be to in use. If there is no work then the repair crew will be move
to a different job, so will not repair the equipment and thus the MC
will never go from under repair to idle awaiting more work.
If the equipment is idle then it cannot breakdown, so it can only change
to being running.
If the equipment is in use then it can change to being either: i) idle
or ii) breakdown so under repair with equal chance.
(a) Write down the transition matrix and draw the transition diagram.
(b) Find the two-step and three-step transition matrices. Further,
write down the (2n) and (2n + 1) transition matrices (i.e. for even
an odd steps).
(c) Use these multi-step transition matrices to explain why this MC
is periodic and define the period.
(d) Suppose the initial state distribution is 0T = [a b c], where a, b, c >
0 and a + b + c = 1. Find the (2n) and (2n + 1) state distributions.
(e) Use the result from (d) to explain whether the limiting distribution exists and if it is unique. You should also make reference to
appropriate theorems in the lecture notes, if there are any.

(f) Find the stationary distribution of the MC.

(g) Using the long term behaviour of the chain to explain why the
stationary distribution exists and is unique, but the limiting distribution cannot exist.

Suppose that we have a Markov chain with state space S =

{0, 1, 2} and transition matrix given by:

1/4 3/4 0
P = 1/3 1/3 1/3
0 1/4 3/4
(a) Compute P (X 0 = 0,X1 = 1, X2 = 1) given the initial state distribution 0T = 41 12 14
(b) Use the theorems in the lecture notes to explain whether the stationary distribution exists and is unique, and do the same for the
limiting distribution. Make statements as to explain why the conditions underlying the theorem conditions are true in this case.
For example, just stating the MC is irreducible is not sufficient
(c) Find the stationary distribution for the MC.

Q3. Consider an Ehrenfest chain with 6 particles, and the following initial

(a) Write down the transition matrix and draw the transition diagram.
(b) If the chain starts with 2 particles in the left partition, write down
the state distribution at the first time step.
(c) Find the stationary distribution using the detailed balance condition.