Uluwwul Himmah

Extraordinary Bonus Home tasks

Q1) A Markov chain X0, X1, X2, on states 0,1,2,3 has the transition probability matrix 0 0.3 0.3 0.4 = 0.5 0 0.4 0.1 0.3 0.4 0 0.3 0.4 0.1 0.5 0

and initial distribution Pr{ X0=0}=0.1, Pr{ X0=1}=0.2, Pr{ X0=2}=0.3, Pr{ X0=3}=0.4 a) b) c) d) e) f) g) h) Determine Pr{ X0=0, X1=2} Determine Pr{ X0=0, X1=2, X2= 1} Determine Pr{X1 =3} Determine Pr{X2 =3} Determine the conditional probabilities: Pr{X1=2, X2= 1| X0=0 } Pr{X1=2, X2= 0| X0=1 } Pr{X2=2, X3= 1| X1=0 } Pr{X2=2, X4= 1| X1=3 } (2 marks) (2 marks) (2 marks) (2 marks) (3 marks) (3 marks) (3 marks) (3 marks)

Q2)

Suppose Xn is a two-state Markov chain whose transition probability matrix is = 0.4 0.6 0.8 0.2

Then Zn=( Xn-1, Xn) is a Markov chain having the four states (0,0),(0,1),(1,0), and (1,1). Determine the transition probability matrix. (10marks) Q3) An urn initially contains a single red ball and a single green ball. A ball is drawn at random, removed, and replaced by a ball of the opposite color, and this process repeats so that there are always exactly two balls in the urn. Let Xn be the number of red balls in the urn after n draws, with X0=1. Specify the transition probabilities for the Markov chain {Xn}. (10marks) A Markov chain on the states 0,1,2,3 has the transition probability matrix 0.1 0.2 0.3 0.4 0 0.3 0.3 0.4 0 0 0.6 0.4 1 0 0 0

Q4)

Determine the corresponding limiting distribution.

(10marks)
1

Uluwwul Himmah

Q5)

Show that the transition probability matrix is regular and computes the limiting distribution. (10marks) 0 0.5 0.5 0 1/3 1/3 0 0 0.5 0 0.5 0 0.5 0 0 1/3 0 0.5 0.5 0 0 0 0 0.5 0

Q6)

From purchase to purchase, a particular customer switches brands among products A, B, and C according to a Markov chain whose transition probability matrix is 0.6 0.2 0.2 0.1 0.7 0.2 0.1 0.1 0.8

In the long run, what fraction of time does this customer purchase brand B? (10marks) Q7) A Markov chain on states {0,1,2,3} has transition probability matrix 0 0 0.8 0.2 0 0 0.4 0.6 0.7 0.3 0 0 0.2 0.8 0 0

a) show that all states are periodic with period d=2; b) find all communicating classes. Q8) = 1

(7 marks) (7 marks)

A two state Markov chain has the transition probability 1

a) Prove by induction that n-step transition matrix is given by 1 (1 ) = + + + b) Determine the first return distribution
( )

=0

(6 marks)

0, , =

0,
)

= 0|

(5 marks)

c) Verify equation = 0 , when i = 0

( )

With

( )

( ) (

Uluwwul Himmah

Q9) a) Give the definition and examples of stochastic and doubly stochastic matrices. b) Prove that the square of stochastic matrix is also stochastic. c) Prove or disprove that the square of doubly stochastic matrix is also doubly stochastic. d) Prove that for any positive integer n, the n-th power of stochastic matrix is stochastic. (16marks) Q10) A Markov chain X0, X1, X2, on states {0,1,2,3} has the transition probability matrix = 0.2 0.1 0.3 0.4 0.3 0.2 0.1 0.3 0.3 0.4 0.5 0.2 0.2 0.3 0.1 0.1

a) Compute the two-step transition matrix b) Compute the three-step transition matrix Q11)

(4 marks) (4 marks)

A Markov chain on states {0,1,2,3} has transition probability matrix. Determine the limiting distribution. (6 marks) 0.1 0.2 0.4 0.3 0.2 0.3 0.1 0.4 0.3 0.4 0.2 0.1 0.4 0.1 0.3 0.2

Q12)

A Markov chain X0, X1, X2, on states {0,1,2,3} has the transition probability matrix = and initial distribution 0.2 0.1 0.3 0.4 0.3 0.2 0.1 0.3 0.3 0.4 0.5 0.2 0.2 0.3 0.1 0.1

p0=Pr{ X0=0}=0.15, p1=Pr{ X0=1}=0.25, p2=Pr{ X0=2}=0.35, p3=Pr{ X0=3}=0.25 a) Determine the probabilities: i) Pr{X0=0, X1=2, X3= 3, X6= 3} ii) Pr{X3 =1} b) Determine the conditional probabilities: i) Pr{X2=2, X4= 1| X1=0} ii) Pr{X2=2, X5= 2| X1=3}

(3 marks) (3 marks)

(3 marks) (3 marks)

Uluwwul Himmah

Q13)

An urn contains six tags, of which three are red and three green. Two tags are selected from the urn. If one tag is red and the other is green, then the selected tags are discarded and two blue tags are returned to the urn. Otherwise, the selected tags are returned to the urn. This process repeats until the urn contains only blue tags. Let Xn denote the number of red tags in the urn after the n-th draw, with X0 =3. Give the transition probability matrix. (6 marks) A Markov chain on states {0,1,2,3,4,5} has transition probability matrix 0.2 0 0 0.8 0 0 0 0.3 0 0 0.7 0 0 0 0.5 0 0 0.5 0.6 0 0 0.4 0 0 0 1 0 0 0 0 0 0 0.3 0 0.3 0.4

Q14)

a) Determine the communicating classes and period for each state. (5 marks) b) Which states are transient and which are recurrent? (5 marks) Q14) A Markov chain on the states { 0,1,2 } has the transition probability matrix 0.3 0.3 0.4 0.2 0.5 0.3 0.4 0.2 0.4

After a long period of time, you observe the chain and see it is in state 0. What is the conditional probability that the previous state was state 1? That is, find (6marks) lim

= 1|

=0

Q15)

Let X be a random variable having the Poisson distribution with parameter . Evaluate the fifth moment E[X5]. (6 marks) From purchase to purchase, a particular customer switches brands among products A, B, and C according to a Markov chain whose transition probability matrix is 0.7 0.1 0.2 0.1 0.8 0.1 0.1 0.1 0.8

Q16)

In the long run, what fraction of time does this customer purchase brand C? (6 marks) Q17) Defects occur along an undersea cable according to a Poisson process of rate =0.3. a) What is the probability that five defects appear in the first 10 miles of cable? (2mrks)
4

Uluwwul Himmah

b) Given that there are 5 defects in the first 10 miles of cable, what is the conditional probability of no defects between mile points 10 and 10.5? (3 marks) c) What is the probability that six defects appear in the first 10.3 miles of cable, seven defects appear in the first 11.4 miles and ten defects appear in the first 11.8 miles of cable? (5 marks) Q18) Let ( ); 0 be a Poisson process having rate parameter =2.

Determine the following expectations: a) E[X(2008)] b) E[X(1) X(2)] Q19)

(2 marks) (4 marks)

Suppose that customers arrive at a facility according to a Poisson process having rate =3. Let X(t) be the number of customers that have arrived up to time t. Determine the following conditional probabilities: a) Pr{X(2)=5| X(4)=9} b) Pr{X(4)=9| X(2)=5} (4 marks) (4 marks)

Q20)

Let 1 and 2 be two stochastic matrices of the same size. Prove that their mean is the stochastic matrix. (5 marks) 0.2 0.1 0.2 0.1 0.3 0.4 0.3 0.4 0.3 0.4 0.3 0.4 0.2 0.1 0.2 0.1

Q21)

A Markov chain X0, X1, X2, on states {0,1,2,3} has the transition probability matrix =

Q22)

a) Compute the two-step transition matrix b) Compute the three-step transition matrix = and initial distribution 0.2 0.1 0.2 0.1 0.3 0.4 0.3 0.4

(4 marks) (6 marks)

A Markov chain X0, X1, X2, on states {0,1,2,3} has the transition probability matrix 0.3 0.4 0.3 0.4 0.2 0.1 0.2 0.1

Pr{ X0 = 0} = 0.15, Pr{ X0 = 1} = 0.25, Pr{ X0 = 2} = 0.35.

Uluwwul Himmah

a) Determine the probabilities: i) Pr{X0= 0, X1= 1, X3= 2, X6= 3} ii) Pr{X3 =3} b) Determine the conditional probabilities: i) Pr{X2=3, X4= 1| X1= 0} ii) Pr{X2=2, X5= 1| X1= 3} Q23)

(4 marks) (5 marks) (3 marks) (3 marks)

An urn initially contains two red balls and two green balls. A ball is drawn at random, removed, and replaced by a ball of the opposite color, and this process repeats so that there are always exactly four balls in the urn. Let Xn be the number of red balls in the urn after n draws, with X0 =2. Specify the transition probability matrix for the Markov chain { Xn } (10marks) A Markov chain on states {0,1,2,3,4,5} has transition probability matrix 0.4 0 0 0.6 0 0 0 0.6 0 0 0.4 0 0 0 0.4 0 0 0.6 0.6 0 0 0.4 0 0 0 0.4 0 0 0.6 0 0 0 0.6 0.1 0.2 0.1

Q24)

a) Determine the communicating classes and period for each state. (5 marks) b) Which states are transient and which are recurrent? (5 marks) Q25) A Markov chain on states {0,1,2,3} has transition probability matrix 0.1 0.3 0.4 0.2 0.4 0.2 0.1 0.3 0.1 0.3 0.4 0.2 0.4 0.2 0.1 0.3

Determine the limiting distribution. Q26)

(10marks)

Let X be a random variable having the Poisson distribution with parameter . Evaluate the third moment E[X3]. (10marks) Defects occur along an undersea cable according to a Poisson process of rate = 0.6. a) What is the probability that three defects appear in the first 10 miles of cable? (4mrk) b) Given that there are 3 defects in the first 10 miles of cable, what is the conditional probability of no defects between mile points 10 and 12? (6 marks)

Q27)

Uluwwul Himmah

Q28)

Suppose that customers arrive at a facility according to a Poisson process having rate = 3. Let X(t) be the number of customers that have arrived up to time t. Determine the following conditional probabilities: a) Pr{X(2) = 5| X(4) = 9} b) Pr{X(4) = 9| X(2) = 5} (4 marks) (6 marks) ( ) , = 1,2, and ( )=

Q29)

a) Using

give the explicit formula for P1(t), P2(t) and P3(t).

( )=

(6 marks)

b) A pure birth process starting from X(0) = 0 has birth parameters 0 = 2, 1= 1, 2= 3, 3= 5. Determine Pr {X(t) = n } for n = 0,1,2,3. (4 marks)