<jupyter_start><jupyter_text>Markov Chain---Suppose there are two possible states in a system (e.g., expansion and recession in the economy) and define$$ p_{1,t} = \Pr\{\text{State 1 in period } t \},\quad p_{2,t} = \Pr\{\text{State 2 in period } t \},$$and$$ \pi_{ij} = \Pr\{\text{State } i \text{ in period } t+1|\text{State } j \text{ in period } t\}.$$Then the Markov chain is defined as$$ \begin{bmatrix} p_{1,t+1} \\ p_{2,t+1} \end{bmatrix} = \begin{bmatrix}  \pi_{11} & \pi_{12} \\  \pi_{21} & \pi_{22}  \end{bmatrix} \begin{bmatrix} p_{1,t} \\ p_{2,t} \end{bmatrix}.$$In general, the Markov chain with $k$ states is given by$$ \underbrace{\begin{bmatrix} p_{1,t+1} \\ \vdots \\ p_{k,t+1} \end{bmatrix}}_{p_{t+1}} = \underbrace{\begin{bmatrix}  \pi_{11} & \cdots & \pi_{1k} \\  \vdots & \ddots & \vdots \\  \pi_{k1} & \cdots & \pi_{kk}  \end{bmatrix}}_{\Pi} \underbrace{\begin{bmatrix} p_{1,t} \\ \vdots \\ p_{k,t} \end{bmatrix}}_{p_t}.$$--- Numerical Example$$ \Pi = \begin{bmatrix}  0.9 & .0.25 \\  0.1 & 0.75  \end{bmatrix},\quad p_0 = \begin{bmatrix} 0.5 \\ 0.5 \end{bmatrix}.$$<jupyter_code>transition_matrix = np.array([[0.9, 0.25], [0.1, 0.75]])
print(transition_matrix)
initial_probability = np.array([0.5, 0.5])
print(initial_probability)<jupyter_output>[0.5 0.5]<jupyter_text>The Markov chain evolves as$$ p_{t+1} = \Pi p_t,\quad t = 0,1,\dots,$$<jupyter_code>markov_chain = np.empty((11,2))
state_probability = initial_probability
markov_chain[0, :] = state_probability
for t in range(1,11):
    state_probability = transition_matrix @ state_probability
    markov_chain[t, :] = state_probability
print(markov_chain)
fig = plt.figure(num=1, facecolor='w')
periods = np.linspace(1, markov_chain.shape[0], markov_chain.shape[0])
plt.plot(periods, markov_chain[:, 0], 'b-x', label='State 1')
plt.plot(periods, markov_chain[:, 1], 'r:.', label='State 2')
plt.xlim((periods.min(), periods.max()))
plt.ylim((0, 1))
plt.title('Markov Chain')
plt.xlabel('Periods')
plt.ylabel('Probability')
plt.legend(loc='best', frameon=False)
# plt.savefig('ms_fig_markovchain.png', dpi=600)
plt.show()
print(markov_chain.shape)
markov_chain.shape[0]<jupyter_output><empty_output><jupyter_text>--- Stationary Distribution of a Markov ChainThe stationary distribution $p^*$ satisfies$$ p^* = \Pi p^*.$$In case of $k=2$, we can derive an explicit formula of $p^*$:$$ p_1^* = \frac{1-\pi_{22}}{2-\pi_{11}-\pi_{22}},\quad p_2^* = \frac{1-\pi_{11}}{2-\pi_{11}-\pi_{22}}.$$<jupyter_code>stationary_distribution = np.array([1.0 - transition_matrix[1, 1], 1.0 - transition_matrix[0, 0]]) \
                          / (2.0 - transition_matrix[0, 0] - transition_matrix[1, 1])
print(stationary_distribution)<jupyter_output>[0.71428571 0.28571429]
{'fraction_non_alphanumeric': 0.21258690084156603, 'fraction_numerical': 0.049762166117819244, 'mean_word_length': 4.637113402061856, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 11, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '142569', 'license': 'MIT', 'n_tokens_mistral': 1158, 'n_tokens_neox': 1053, 'n_words': 219, 'path': 'notebook-b/markovchain.ipynb', 'repo_name': 'nakatsuma/probability_and_statistics', 'size': '2733'}