Show that $\{(X_{n},X_{n+1})\}_{n\geq 0}$ is a Markov Chain where $\{X_{n}\}_{n\geq 0}$ is a Markov Chain Show that $\{(X_{n},X_{n+1})\}_{n\geq 0}$ is a Markov Chain where $\{X_{n}\}_{n\geq 0}$ is a Markov Chain.
Remark: We know that $\mathbb{P}(A|\emptyset)$ is undefined, I am right?
This fact is important in my attempt and is what motivates this post.
My attempt: We define $Y_{n}:=(X_{n},X_{n+1})$, then we are going to verify if we have the Markov property. On the one hand, we know that
\begin{align}
\mathbb{P}[Y_{n}=(i,j)\:|\:Y_{n-1}=(i_{n-1},j_{n-1})]&=\mathbb{P}[X_{n}=i,X_{n+1}=j\:|\:X_{n-1}=i_{n-1},X_{n}=j_{n-1}]\\
&=\left\{\begin{array}{ll}0 &\mbox{ if }i\neq j_{n-1}\\ p_{ij} &\mbox{ if } i=j_{n-1}. \end{array}\right.
\end{align}
where $P$ is transition matrix of $\{X_{n}\}_{n\geq 0}$. On the other hand, we have
\begin{align}
&\mathbb{P}[Y_{n}=(i,j)\:|\:Y_{0}=(i_{0},j_{0}),\ldots,Y_{n-2}=(i_{n-2},j_{n-2}),Y_{n-1}=(i_{n-1},j_{n-1})]\\
&=\mathbb{P}\left[X_{n}=i,X_{n+1}=j\:|\:X_{0}=i_{0},X_{1}=j_{0},\ldots,X_{n-2}=i_{n-2},X_{n-1}=j_{n-2},X_{n-1}=i_{n-1},X_{n}=j_{n-1}\right]\\
&=\left\{\begin{array}{ll}\mbox{undefined} & \mbox{if }\exists k\in\{0,1,\ldots,n-2\}\mbox{ such that }j_{k}\neq i_{k+1}\\ 0 & \mbox{if }\forall k\in\{0,1,\ldots,n-2\}\mbox{ such that }j_{k}=i_{k+1} \mbox{ and }j_{n-1}\neq i\\ p_{ij} & \mbox{if }\forall k\in\{0,1,\ldots,n-2\}\mbox{ such that }j_{k}=i_{k+1} \mbox{ and }j_{n-1}= i \end{array}\right.
\end{align}
My problem is in the undefined case, this is making me think that this chain is not a Markov chain. What do you think about this?
This exercise appears in Adventures in Stochastic Processes; Resnick, Sidney I.

Usually, in the definition of a Markov chain (with state space $\mathbb N$),
we require the equality
$$
\mathbb P\left(X_n=j\mid X_{n-1}=i,X_{k}=i_k,0\leqslant k\leqslant n-2\right)=\mathbb P\left(X_n=j\mid X_{n-1}=i\right)
$$
to hold for each $i$, $i_1,\dots,i_{n-2}$ such that the event $A:=\{X_{n-1}=i,X_{k}=i_k,0\leqslant k\leqslant n-2\}$ has a positive probability. This is possible that the previously event $A$ has probability zero, for example if $i_1$ and $i_2$ are two states which do not communicate. So an undefined case may also hold for Markov chains which are not defined as a vector like $(X_n,X_{n+1})$ hence this "issue" (quote mark because it is not really one) is not specific to this problem.
