Spring 2018, Math 171
Week 2
-
Markov/Non-Markov Chains
-
(Discussed) Example 1.2 from the book (Ehrenfest Chain)
-
(Discussed) At \(t=0\) an urn contains \(N\) balls, \(M\) of which are red, \(N-M\) of which are green. Each day (\(t = 1, 2, \dots\)) a ball is drawn without replacement. Let \(X_n\) be the color of the ball drawn at \(t=n\). Is \(\{X_n:N \ge n \ge 1\}\) a Markov Chain? Prove your claim.
-
Example 1.6 from the book (Inventory Chain)
-
Let \(\{X_n:n \ge 0\}\) be a Markov Chain on the state space \(\mathcal{S}=\{0, 1, 2\}\). Define \[Y_n=I_{[X_n \ge 1]} = \begin{cases}1 &\text{if } X_n =1,2 \cr 0 &\text{if } X_n = 0\end{cases}\] Under what circumstances, if any, is \(\{Y_n:n \ge 0\}\) a Markov Chain?
-
-
Stopping/Non-Stopping Times
-
Let \(\{X_n:n \ge 0\}\) be a Markov Chain. Which of the following will necessarily be stopping times? Prove your claims.
- (Discussed) \(T=\min\{n \ge 0: X_n = x\}\)
- (Discussed) \(T=\max\{n \ge 0: X_n = x\}\)
- \(T=\min\{n \ge 0: X_n = X_{n-1}\}\)
- (Discussed) \(T=\min\{n \ge 0: X_{n+1} = X_{n}\}\)
-
Let \(T_1, T_2\) be stopping times for some Markov Chain \(\{X_n:n \ge 0\}\). Which of the following will also necessarily be stopping times? Prove your claims.
-
\(T_1 + T_2\)
- (Answer) Yes
-
\(T_1 - T_2\)
- (Answer) No
-
\(\min(T_1, T_2)\)
- (Answer) Yes
-
\(\max(T_1, T_2)\)
- (Answer) Yes
-
-
-
Linear Algebra
-
Compute or write down the inverse of the matrix \[A = \begin{bmatrix} a & b \cr c & d \end{bmatrix}\]
- (Answer) \[\frac{1}{ad-bc}\begin{bmatrix}d & -b \\ -c & a\end{bmatrix}\]
-
Under what circumstances is the following matrix invertible? Under these circumstances, compute its inverse. \[A = \begin{bmatrix} a & 0 & 0 \cr b & c & 0 \cr 0 & d & e \end{bmatrix}\]
- (Answer) \(ace \neq 0\)\[\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}\]
-
Compute a left eigenvector of \[P = \begin{bmatrix} 1-r & 0 & r \cr p & 1-p & 0 \cr 0 & q & 1-q \end{bmatrix}\] corresponding to eigenvalue \(1\)
- (Answer) \(ace \neq 0\)\[\begin{bmatrix}1/a & 0 & 0 \\ -b/ac & 1/c & 0\\bd/ace & -d/ce & 1/e\end{bmatrix}\]
-