Markov Chains - understand proof that if x and y communicate then if x is recurrent then y must also be recurrent I am trying to understand the proof that if two states that communicate, then if one state is recurrent the other must also be recurrent.
The book I'm looking at has this proof:
Suppose $x$ is recurrent, and that $y$ communicates with $x$. So there exist integers $k,l \geq 1$ such that $p_k(x,y)>0$ and $p_l(y,x)>0$.
By Chapman-Kolmogorov, $p_{k+n+l}(y,y)\geq p_{l}(y,x)p_{n}(x,x)p_{k}(x,y)$
So that's my first question - how do we derive the above from the Chapman-Kolmogorov equations?
The proof goes on: so by the recurrence of x and a previous theorem stating that state $x$ is recurrent if and only if the expected number of visits to $x$ is infinite, $\sum ^\infty _{n=0}p_n(y,y) \geq p_{l} (y,x)p_{k}(x,y)\sum^\infty _{n=0}p_n(x,x)=\infty$
My second question is - is what allow us to move from the previous inequality to those summations (especially the first summation where we're going from $p_{k+n+l}(y,y)$ to $\sum^\infty _{n=0}p_n(y,y)$
Thank you. 

Regarding $p_{k+n+l}(y,y) \ge p_l(y,x) p_n(x,x) p_k(x,y)$ can be intuitively explained as follows:


*

*The left-hand side is the probability of starting at $y$ and after $k+n+l$ steps landing at $y$ again.

*The right-hand side is the probability of starting at $y$, then after $l$ steps landing at $x$, then after $n$ more steps landing at $x$ again, and then after $k$ more steps landing at $y$.

*The paths considered by the right-hand side each go from $y$ to $y$ in $l+n+k$ steps, but there are many other ways to do so, so the probability on the right-hand side is smaller.


More succinctly,
\begin{align}
&P(\text{go from $y$ to $y$ in $k+n+l$ steps})
\\
&= \sum_{a,b} P(\text{go from $y$ to $a$ in $k$ steps, then to $b$ in $n$ steps, then to $y$ in $l$ steps})
\\
&\ge P(\text{go from $y$ to $x$ in $k$ steps, then to $x$ in $n$ steps, then to $y$ in $l$ steps}).
\end{align}

For the second question,
$$\sum_{j=0}^\infty p_j(y,y) \ge \sum_{j=l+k}^\infty p_j(y,y) \ge p_l(y,x) p_k(x,y) \sum_{n=0}^\infty p_n(x,x),$$
where the last step comes from writing $j=l+n+k$.
