Transition matrix (Markov chain) mean values If I have a n $\times$ n transition matrix and want to find the expected times passing the point j if the starting point is i and the amount of steps is m.

For example
$$P =\begin{bmatrix}0.5&amp;0.5&amp;0&amp;0&amp;0\\0.5&amp;0&amp;0.5&amp;0&amp;0\\0.5&amp;0&amp;0&amp;0.5&amp;0\\0&amp;0&amp;0.5&amp;0&amp;0.5\\0&amp;0.5&amp;0.5&amp;0&amp;0\end{bmatrix}$$

i = 1, 
 j = 5, 
 m = 100 

Thank you.
The transition probabilities after $k$ steps become $\mathbf{P}^k$. Hence, the $(i, j)$ entry of matrix $\mathbf{N}$ below is the expected number of times the chain is in state $j$, given that the chain started in state $i$.

$$\mathbf{N}=\sum_{k=0}^m{\mathbf{P}^k}$$


If the matrix $\mathbf{I-P}$ is invertible, then the above sum can be expressed as $(\mathbf{I}-\mathbf{P}^{m+1})(\mathbf{I}-\mathbf{P})^{-1}$ using the geometric sum formula. 
Otherwise, if $\mathbf{P}$ is diagonalizable
$$\mathbf{P}=\mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1}$$
where $\mathbf{\Lambda}$ is a diagnoal matrix with elements $\lambda_i$ that are eigen values of $\mathbf{P}$. The matrix $\mathbf{N}$ is then can be given by
$$\mathbf{N}=\mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$$
where $\mathbf{D}$ is a diagonal matrix whose diagonal elements are given by $$d_i=\begin{cases}m+1,&amp;\lambda_i=1\\\frac{1-\lambda_i^{m+1}}{1-\lambda_i},&amp;\lambda_i\neq1\end{cases}$$
The transition probabilities after $k$ steps become $\mathbf{P}^k$. Hence, the $(i, j)$ entry of matrix $\mathbf{N}$ below is the expected number of times the chain is in state $j$, given that the chain started in state $i$.

$$\mathbf{N}=\sum_{k=0}^m{\mathbf{P}^k}$$


If the matrix $\mathbf{I-P}$ is invertible, then the above sum can be expressed as $(\mathbf{I}-\mathbf{P}^{m+1})(\mathbf{I}-\mathbf{P})^{-1}$ using the geometric sum formula. 
Otherwise, if $\mathbf{P}$ is diagonalizable
$$\mathbf{P}=\mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1}$$
where $\mathbf{\Lambda}$ is a diagnoal matrix with elements $\lambda_i$ that are eigen values of $\mathbf{P}$. The matrix $\mathbf{N}$ is then can be given by
$$\mathbf{N}=\mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$$
where $\mathbf{D}$ is a diagonal matrix whose diagonal elements are given by $$d_i=\begin{cases}m+1,&amp;\lambda_i=1\\\frac{1-\lambda_i^{m+1}}{1-\lambda_i},&amp;\lambda_i\neq1\end{cases}$$