Calculating conditional probability for markov chain I have a Markov chain with state space $E = \{1,2,3,4,5\}$ and transition matrix below:
$$ \begin{bmatrix} 1/2 & 0 & 1/2 & 0 & 0 \\\ 1/3 & 2/3 & 0 & 0 & 0 \\\ 0 & 1/4 & 1/4 & 1/4 & 1/4 \\\ 0 & 0 & 0 & 3/4 & 1/4 \\\ 0 & 0 & 0 & 1/5 & 4/5\ \end{bmatrix} $$
How would I find the conditional probabilities of $\mathbb{P}(X_2 = 5 | X_0 =1)$ and $\mathbb{P}(X_3 = 1 | X_0 =1)$?
I am trying to use the formula (or any other formula, if anyone knows of any) $p_{ij}^{(n)} = \mathbb{P}(X_n = j | X_0 =i)$, the probability of going from state $i$ to state $j$ in $n$ steps.
So $\mathbb{P}(X_2 = 5 | X_0 =1) = p_{15}^2$, so I read the entry in $p_{15}$, and get the answer is $0^2$, but the answer in my notes say it is $1/8$?
Also, I get for $\mathbb{P}(X_3 = 1 | X_0 =1) = p_{11}^3 = (\frac{1}{2})^3 = 1/8$, but the answer says it is $1/6$?

I will give not really formal solution, but maybe this will help you too. $P(X_2=5|X_0=1)$ means getting from the state 1, at the moment 0, to the state 5, at the moment 2. So we are allowed to make to make two steps. Final destination - state 5, is column 5, so nonzero probabilities to get there are from states 3,4,5. So the first step must be getting to one of these. Lets check first row - from state 1 we can get only to states 1,3, which leaves us only one way $1\rightarrow 3\rightarrow 5$, with probability $1/2\cdot1/4=1/8$. A bit longer different situation is $P(X_3=1|X_0=1)$, here we have three steps. Thinking in the same way as before there are two ways with probabilities $(1/2)^3$ and $1/3\cdot1/4\cdot1/2$ (sums to $1/6$). Of course Didier's solution is more formal, and it may be worth doing it in this my way only when we have lots of zeros, like in this case (to eliminate possible combinations). This should help you understanding why $p_{15}^2$ doesn't work here - that would mean $1\rightarrow 1\rightarrow 1$, staying in state 1.
