# Spring 2018, Math 171

#### Week 2

1. Markov/Non-Markov Chains

1. (Discussed) Example 1.2 from the book (Ehrenfest Chain)

2. (Discussed) At $$t=0$$ an urn contains $$N$$ balls, $$M$$ of which are red, $$N-M$$ of which are green. Each day ($$t = 1, 2, \dots$$) a ball is drawn without replacement. Let $$X_n$$ be the color of the ball drawn at $$t=n$$. Is $$\{X_n:N \ge n \ge 1\}$$ a Markov Chain? Prove your claim.

3. Example 1.6 from the book (Inventory Chain)

4. Let $$\{X_n:n \ge 0\}$$ be a Markov Chain on the state space $$\mathcal{S}=\{0, 1, 2\}$$. Define $Y_n=I_{[X_n \ge 1]} = \begin{cases}1 &\text{if } X_n =1,2 \cr 0 &\text{if } X_n = 0\end{cases}$ Under what circumstances, if any, is $$\{Y_n:n \ge 0\}$$ a Markov Chain?

2. Stopping/Non-Stopping Times

1. Let $$\{X_n:n \ge 0\}$$ be a Markov Chain. Which of the following will necessarily be stopping times? Prove your claims.

1. (Discussed) $$T=\min\{n \ge 0: X_n = x\}$$
2. (Discussed) $$T=\max\{n \ge 0: X_n = x\}$$
3. $$T=\min\{n \ge 0: X_n = X_{n-1}\}$$
4. (Discussed) $$T=\min\{n \ge 0: X_{n+1} = X_{n}\}$$
2. Let $$T_1, T_2$$ be stopping times for some Markov Chain $$\{X_n:n \ge 0\}$$. Which of the following will also necessarily be stopping times? Prove your claims.

1. $$T_1 + T_2$$

2. $$T_1 - T_2$$

3. $$\min(T_1, T_2)$$

4. $$\max(T_1, T_2)$$

1. Compute or write down the inverse of the matrix $A = \begin{bmatrix} a & b \cr c & d \end{bmatrix}$
• (Answer) $\frac{1}{ad-bc}\begin{bmatrix}d & -b \\ -c & a\end{bmatrix}$
2. Under what circumstances is the following matrix invertible? Under these circumstances, compute its inverse. $A = \begin{bmatrix} a & 0 & 0 \cr b & c & 0 \cr 0 & d & e \end{bmatrix}$
• (Answer) $$ace \neq 0$$$\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}$
3. Compute a left eigenvector of $P = \begin{bmatrix} 1-r & 0 & r \cr p & 1-p & 0 \cr 0 & q & 1-q \end{bmatrix}$ corresponding to eigenvalue $$1$$
• (Answer) $$ace \neq 0$$$\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}$