Spring 2018, Math 171

Week 2

  1. Markov/Non-Markov Chains

    1. (Discussed) Example 1.2 from the book (Ehrenfest Chain)

    2. (Discussed) At \(t=0\) an urn contains \(N\) balls, \(M\) of which are red, \(N-M\) of which are green. Each day (\(t = 1, 2, \dots\)) a ball is drawn without replacement. Let \(X_n\) be the color of the ball drawn at \(t=n\). Is \(\{X_n:N \ge n \ge 1\}\) a Markov Chain? Prove your claim.

    3. Example 1.6 from the book (Inventory Chain)

    4. Let \(\{X_n:n \ge 0\}\) be a Markov Chain on the state space \(\mathcal{S}=\{0, 1, 2\}\). Define \[Y_n=I_{[X_n \ge 1]} = \begin{cases}1 &\text{if } X_n =1,2 \cr 0 &\text{if } X_n = 0\end{cases}\] Under what circumstances, if any, is \(\{Y_n:n \ge 0\}\) a Markov Chain?

  2. Stopping/Non-Stopping Times

    1. Let \(\{X_n:n \ge 0\}\) be a Markov Chain. Which of the following will necessarily be stopping times? Prove your claims.

      1. (Discussed) \(T=\min\{n \ge 0: X_n = x\}\)
      2. (Discussed) \(T=\max\{n \ge 0: X_n = x\}\)
      3. \(T=\min\{n \ge 0: X_n = X_{n-1}\}\)
      4. (Discussed) \(T=\min\{n \ge 0: X_{n+1} = X_{n}\}\)
    2. Let \(T_1, T_2\) be stopping times for some Markov Chain \(\{X_n:n \ge 0\}\). Which of the following will also necessarily be stopping times? Prove your claims.

      1. \(T_1 + T_2\)

        • (Answer) Yes
      2. \(T_1 - T_2\)

        • (Answer) No
      3. \(\min(T_1, T_2)\)

        • (Answer) Yes
      4. \(\max(T_1, T_2)\)

        • (Answer) Yes
  3. Linear Algebra

    1. Compute or write down the inverse of the matrix \[A = \begin{bmatrix} a & b \cr c & d \end{bmatrix}\]

      • (Answer) \[\frac{1}{ad-bc}\begin{bmatrix}d & -b \\ -c & a\end{bmatrix}\]
    2. Under what circumstances is the following matrix invertible? Under these circumstances, compute its inverse. \[A = \begin{bmatrix} a & 0 & 0 \cr b & c & 0 \cr 0 & d & e \end{bmatrix}\]

      • (Answer) \(ace \neq 0\)\[\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}\]
    3. Compute a left eigenvector of \[P = \begin{bmatrix} 1-r & 0 & r \cr p & 1-p & 0 \cr 0 & q & 1-q \end{bmatrix}\] corresponding to eigenvalue \(1\)

      • (Answer) \(ace \neq 0\)\[\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}\]