Probability $\mathbb{P}[X_N = 1 | X_0 = 1]$ for a Markov Chain over $\mathbb{X} = \{1,2,3\}$ Exercise :


  Let $\{ X_n \}_{n \in \mathbb{N}_0}$ be a Markov Chain over the space $\mathbb{X} = \{ 1,2,3 \}$ with the following transition matrix :
  $$P = \begin{pmatrix} 0 &amp; 1/2 &amp; 1/2 \\ 1/3 &amp; 1/3 &amp; 1/3 \\ p &amp; 2/3 -p &amp; 1/3 \end{pmatrix}$$
  Calculate the probability $\mathbb{P}[X_n=1|X_0=1]$ for all $n \in \mathbb{N}$ for the cases of a) $p=0$, b) $p =1/6$, c) $p=2/3$. How would you calculate $\lim_{n\to \infty} P^n$ without many operations ?


Attempt :

First of all, I start by finding the eigenvalues and eigenvectors for $P$, which are : 

$$\det(P-\lambda I) = 0 \Rightarrow \begin{vmatrix} 0 &amp; 1/2 &amp; 1/2 \\ 1/3 &amp; 1/3 &amp; 1/3 \\ p &amp; 2/3 -p&amp; 1/3 \end{vmatrix} = 0 \Leftrightarrow \dots \Leftrightarrow \lambda = \begin{cases} 1 \\ \frac{1}{6}\big(\pm\sqrt{6p-1}-1\big)\end{cases}$$

The problem I have though, is that for example, the case $p=0$ yields complex eigenvalues, which I don't know how to handle in terms of probabilities to continue on. Also, in the case $p=1/6$ we have a Jordan case, which still leaves us with a hard case for $P^n$. Any tips on how to calculate the given probability ?
I would guess its something like this:

$x_{n}$ = $A^{n}$ * $x_{0}$

$x_{n}$ = ($P^{-1} L^{n} P$) * $x_{0}$

Where P is the matrix made of all the eigenvectors of A, L is the matrix with the corresponding eigenvalues, and $x_{0}$ is the unit vector [1, 0, 0].

Since you only care about the probability Pr($X_{n}$ = 1 | $X_{0}$ = 1), you only care about the element $A^{n}_{0,0}$. Since you have complex eigenvectors, your final result should be real positive number.
I would guess its something like this:

$x_{n}$ = $A^{n}$ * $x_{0}$

$x_{n}$ = ($P^{-1} L^{n} P$) * $x_{0}$

Where P is the matrix made of all the eigenvectors of A, L is the matrix with the corresponding eigenvalues, and $x_{0}$ is the unit vector [1, 0, 0].

Since you only care about the probability Pr($X_{n}$ = 1 | $X_{0}$ = 1), you only care about the element $A^{n}_{0,0}$. Since you have complex eigenvectors, your final result should be real positive number.