Randomly swapping two balls in two urns with 3 balls each. In total $3$ are black and 3 are white. Is this process a Markov chain? Three white and three black balls are distributed in two urns in such a way that
each contains three balls. We say that the system is in state i, i = 0, 1, 2, 3, if the first urn contains i white balls. At each step, we draw one ball from each urn and place the ball drawn from the first urn into the second, and conversely with the ball from the second urn.
Let $X_n$ denote the state of the system after nth step. Now how to prove that $(X_n=0,1,2,...)$is a markov chain and how to calculate its transition probability matrix.
Solution:If at the initial stage both the urns have three balls each and we draw one ball from each urn and place them into urn different from the urn from which it is drawn. So after nth step state of the system will be 3 and it will remain it forever. So this is not a markov chain. I also want to understand the meaning of bold line.
If I am wrong, explain me why and how I am wrong and what is the transition matrix of this markov chain. Would any one answer this question?  

Define $F_n$ to be the indicator random variable which has value 1 if at $n^{th}$ step white ball Chosen from the first ball and else 0.
Similarly Define Indicator random variable $S_n$ for second urn
Now,To check Markov property we need to check 
P($X_n$= j|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))=P($X_n$= j|$X_{n-1}$=$i_{n-1}$)
First Observe the conditional Range of $X_n$ given $X_{n-1}$,$X_{n-2}$...,$X_0$ is 
$\space$ {$X_{n-1}-1$, $X_{n-1}$, $X_{n-1}+1$}
Hence enough to check for these cases below.
if j = $i_{n-1}+1$
Then rewrite this prob as P($F_n$=0,$S_n$=1|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))
if j= $i_{n-1} -1$
Then rewrite this prob as P($F_n$=1,$S_n$=0|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))
if j = $i_{n-1}$
Then rewrite this prob as 
P($F_n$=1,$S_n$=1|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))+ P($F_n$=0,$S_n$=0|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))
Now,Observe that $F_n$ and $S_n$ only depends on $X_{n-1}$ and not on $X_{n-2}$,$X_{n-3}$,...$X_0$
So,from here you can conclude 
P($X_n$= j|($X_{n-1}$,$X_{n-2}$,..$X_0$)=($i_{n-1}$,$i_{n-2}$,...$i_0$))=
P($X_n$= j|$X_{n-1}$=$i_{n-1}$)
$\textbf{NOTE:}$ This is a long hand mathematical approach to argue markov property of $X_n$.If one wants to keep life simple one can argue in one line that the $n^th$ draw depends only on the $X_{n-1}$ by giving appropriate arguments.
