Solving a Markov Chain Let the distribution on variables $(X_t)$ for $t \in N$ satisfy a Markov chain. Each variable can take the values $\{1, 2\}$. We are given the pmfs
$$p(X_1=i) = 0.5$$
for $i=1,2$ and
$$p(X_{t+1} = j\mid X_t = i) = p_{i,j}$$
where $p_{i,j}$ is the $(i, j)$-th element of the matrix
$$P=\begin{pmatrix}
0.3 & 0.7\\
0.6 & 0.4
\end{pmatrix}$$
Find: $P(X_3 = 2)$ and $p(X_2 = 1\mid X_3 = 2)$.
I'm stuck with how to start this problem. So any hints would be appreciated.  

Hints: By the law of total probability: 
$$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_1=i,X_2=j)P(X_1=i,X_2=j)$$ But by the Markov property, $X_3$ is independent of $X_1$, hence the inner term simplifies to $$P(X_3=2)=\sum_{i,j\in \{1,2\}}P(X_3=2\mid X_2=j)P(X_1=i,X_2=j)$$ Now, $P(X_3=2\mid X_2=j)$ is easy to compute (right?) and $$P(X_1=i,X_2=j)=P(X_2=j\mid X_1=i)P(X_1=i)$$ which is again easy to compute (right?) for any $i,j$. 
For the second part, just use Bayes rule:
$$P(X_2=1\mid X_3=2)=\frac{P(X_3=2\mid X_2=1)P(X_2=1)}{P(X_3=2)}$$ and of course use the first part to avoid repeating calculations.
