Assignment #3 (July, 24)

1. Suppose 4 balls are placed into two urns A and B. On each day, One ball is selected such that each of the four balls is equally likely to be selected and the ball is then placed into the other urn. Let Xn be the number of balls in urn A on the nth day. (a) Is {Xn } n=0 a Markov chain. If it is, write down their state spaces and transition matrices and do the usual classication. (b) Given X0 = 1, nd the probability function of X2 . (c) In a long run, what proportion of times when at least one urn is empty? (d) Given X0 = k , calculate the probability that number of balls in urn A reaches 0 before the number of balls in urn B reaches 0 for k = 0, 1, 2, 3 and 4. 2. Consider the Markov chain with the state space S = {0, 1, 2, 3, 4, 5} and the transition matrix
1 5 0 0 P = 0 0
1 4 1 5 1 3 1 5

0
2 3

1 5

1 5

0
1 2

0
1 2

0
3 5

0
2 5

0
1 2

0
1 2 1 2

0
1 4

0 0

0 0 0 0 0

(a) Show that S consists of two closed classes and one open class. (b) Find the period of each of the three classes. (c) Find the unique equilibrium distribution corresponding to each closed class. (d) Find the probability of absorption into {1, 3} from state 0 and the probability of absorption into {1, 3} from state 5. What can you say about the probabilities of absorption in {2, 4} from states 0 and 5 respectively? (e) Find the equilibrium distribution when X0 = 0. (f) Let T0 be the waiting time for going to the closed class, starting from state 0. Find E (T0 ). 3. Consider a modied symmetric random walk on {0, 1, . . . , N } with reecting barriers at both 0 (reecting to 1 once reaching 0) and N (reecting to N 1 once reaching N ). For intermediate states 1 j N 1, we have Pj,j +1 = Pj,j 1 = 1/2. (a) Argue that the Markov Chain is irreducible and all states are positive recurrent. (b) Find the unique equilibrium distribution of this chain. (c) Find the expected number of steps between returns to 0, and the expected number of steps between returns to 1. 4. Telephone calls arrive at a switchboard in a Poisson process at the rate of 2 per minute. A random one-tenth of the calls are long distance. (a) What is the probability that no call arrives between 9:00-9:05am?

(b) What is the probability that at least 2 calls arrive between 10:00-10:02am? (c) What is the probability of at least one long distance call in a ten minute period? (d) Given that there have been 8 long distance calls in an hour, what is the expected number of calls to have arrived in the same period? (e) Given that there were 90 calls in an hour, what is the probability that 10 were long distance? 5. A bank arrange service as follows: If no more than one customer is in the bank, there will be only one teller at a window. If a second customer enters while the rst is still in service, a second teller will come to another window and begin to service the second customer. Similarly, if a third customer enters while the rst and second are still being serviced, a third teller comes to a window and begins to service the third customer. However, there are only three tellers so any further customers have to wait in line. Assume the service time of each teller is exponentially-distributed with mean 1/ and assume that the time between customer arrivals is exponentially-distributed with mean 1/. (We often say that the customers arrive at a Poisson rate or at an exponential rate .) (a) If one customer is in the bank, what is the expected length of time until a departure? (b) If two customers are in the bank, what is the expected length of time until a departure? (c) If three customers are in the bank, what is the expected length of time until a departure? (d) If ten customers are in the bank, what is the expected length of time until a departure? (e) If x customers are in the bank, what is the expected length of time until an arrival? (f) If all three tellers are busy, what is the probability that the customer at Teller #1 departs rst? (For part (a,b), work under the assumption that there is no new customer arriving before a departure. 6. (Bonus question) A small company supplies power to m similar machines. If a machine is not working at time t, the waiting time for requiring power is exponential distributed with rate . Once a machine is turned on, it operates for an exponentially distributed length of time with rate . Let X (t) be the number of machines operating at time t. Suppose we start with all m machines operating (i.e., X (0) = m). Find P (X (t) = k |X (0) = m) for k = 0, . . . , m. You do not need to work on the last question, but if you would work it out, you will get some bonus mark.