# Spring 2018, Math 171

#### Week 4

1. Stationary Distributions

1. Compute any and all stationary distributions of $P = \begin{bmatrix} P_1 & 0 & \dots & 0 \cr 0 & P_2 & \dots & 0 \cr \vdots & \vdots & \ddots & \vdots \cr 0 & 0 & \dots & P_k \end{bmatrix}$ where $$P_i$$ has a unique stationary distribution $$\pi_i$$ for each $$i$$. If you claim $$P$$ has a unique stationary distribution, please justify.

2. (Discussed) Expanding upon problem 2.1 in the week 3 handout, consider the Markov chain defined by the following transition matrix: $P = \begin{matrix} & \mathbf 1 & \mathbf 2 & \mathbf 3 & \mathbf 4 & \mathbf 5 & \mathbf 6 & \mathbf 7 & \mathbf 8 \cr \mathbf 1 & 0.5 & 0 & 0.5 & 0 & 0 & 0 & 0 & 0 \cr \mathbf 2 & 0.5 & 0.5 & 0 & 0 & 0 & 0 & 0 & 0 \cr \mathbf 3 & 0 & 0 & 0 & 0.5 & 0 & 0 & 0 & 0.5 \cr \mathbf 4 & 0 & 0 & 0.5 & 0 & 0.5 & 0 & 0 & 0 \cr \mathbf 5 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \cr \mathbf 6 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \cr \mathbf 7 & 0 & 0 & 0 & 0 & 0 & 0.5 & 0 & 0.5 \cr \mathbf 8 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \end{matrix}$ Compute any and all stationary distributions. If you claim $$P$$ has a unique stationary distribution, please justify.

2. Reversibility/Detailed Balance Condition

1. Show that the Markov Chain defined by $P = \begin{bmatrix} 1-p & p \cr q & 1-q \end{bmatrix}$ with $$p,q > 0$$ will always satisfy detailed balance

2. Does the following satisfy detailed balance? $P = \begin{bmatrix} 0.8 & 0.1 & 0.1 \cr 0.1 & 0.7 & 0.2 \cr 0.3 & 0.6 & 0.1 \end{bmatrix}$

3. (Discussed) Under what conditions on $$p,q$$ will the following satisfy detailed balance? $P = \begin{bmatrix} 0 & 0.5 & 0.5 \cr p & 1-p-q & q \cr 0.4 & 0.6 & 0 \end{bmatrix}$

3. Limiting Behavior

1. (Discussed) Expanding upon problem 2.1 in the week 3 handout, consider the Markov chain defined by the following transition matrix: $P = \begin{matrix} & \mathbf 1 & \mathbf 2 & \mathbf 3 & \mathbf 4 & \mathbf 5 & \mathbf 6 & \mathbf 7 & \mathbf 8 \cr \mathbf 1 & 0.5 & 0 & 0.5 & 0 & 0 & 0 & 0 & 0 \cr \mathbf 2 & 0.5 & 0.5 & 0 & 0 & 0 & 0 & 0 & 0 \cr \mathbf 3 & 0 & 0 & 0 & 0.5 & 0 & 0 & 0 & 0.5 \cr \mathbf 4 & 0 & 0 & 0.5 & 0 & 0.5 & 0 & 0 & 0 \cr \mathbf 5 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \cr \mathbf 6 & 0 & 0 & 0 & 0 & 0 & 0 & 1 & 0 \cr \mathbf 7 & 0 & 0 & 0 & 0 & 0 & 0.5 & 0 & 0.5 \cr \mathbf 8 & 0 & 0 & 0 & 0 & 0 & 1 & 0 & 0 \end{matrix}$ Compute $$E_2[N(6)]$$ and $$E_2[N(3)]$$

2. Compute $$E_1[N(2)]$$ and $$E_2[N(1)]$$ for the Markov chain defined by the following transition matrix: $P = \begin{matrix} & \mathbf 1 & \mathbf 2 & \mathbf 3 \cr \mathbf 1 & 0.1 & 0.5 & 0.4 \cr \mathbf 2 & 0.4 & 0.2 & 0.4 \cr \mathbf 3 & 0 & 0 & 1 \end{matrix}$