<jupyter_start><jupyter_text>Dynamic Bayesian Network, Markov ChainLet's see how we can represent a [Markov Chain](https://en.wikipedia.org/wiki/Examples_of_Markov_chains) (MC) as a Dynamic Bayesian Network (DBN). We will verify our results with the stationary distribution or steady state.  ModelThe model, $P$, has two states, sunny and rainy. Our initial state, $s$, will be sunny.<jupyter_code>import numpy as np

P = np.array([[0.9, 0.1], [0.5, 0.5]])
s = np.array([1.0, 0.0])<jupyter_output><empty_output><jupyter_text>Power MethodWe can use the [Power Method](https://en.wikipedia.org/wiki/Power_iteration) to find the steady state.$s^n = s^0P^n$We will continue to compute $s^n$ until $||P^n - P^{n-1}|| \lt \delta$.<jupyter_code>s.dot(P)
s.dot(P).dot(P)
import pandas as pd

P_curr = s.dot(P)
trace_df = [list(P_curr) + [np.nan]]

for it in range(100):
    P_next = P_curr.dot(P)
    d = np.linalg.norm(P_next - P_curr, 1)
    
    if d < 0.0001:
        break
    
    trace_df.append(list(P_next) + [d])
    P_curr = P_next
    
trace_df = pd.DataFrame(trace_df, columns=['sunny', 'rainy', 'delta'])
P_next<jupyter_output><empty_output><jupyter_text>Here is the trace of $s^n$ until convergence.<jupyter_code>trace_df<jupyter_output><empty_output><jupyter_text>Eigen decompositionWe can also find the steady state directly through eigen decomposition. The steady state is the eigenvector associated with the eigenvalue closest to 1.<jupyter_code>def get_stationary_p(X):
    S, U = np.linalg.eig(X.T)
    r = (U[:,np.isclose(S, 1)][:,0] / U[:,np.isclose(S, 1)][:,0].sum()).real
    return r

get_stationary_p(P)<jupyter_output><empty_output><jupyter_text>Dynamic Bayesian NetworkAnother way to represent a MC is with a DBN. In general, the DBN structure consists of an endless number of nodes corresponding to time slices. However, in a DBN, we can `unroll` the DBN structure to just 2 consecutive time slices. It is assumed that the intra- (within a time slice) and inter-relationships (between two consecutive time slices) do not change over time.<jupyter_code>from pybbn.graph.dag import Bbn
from pybbn.graph.edge import Edge, EdgeType
from pybbn.graph.jointree import EvidenceBuilder
from pybbn.graph.node import BbnNode
from pybbn.graph.variable import Variable
from pybbn.pptc.inferencecontroller import InferenceController

# create the nodes
w_curr = BbnNode(Variable(0, 'weather', ['rainy', 'sunny']), [1.0, 0.0])
w_next = BbnNode(Variable(2, 'weather_next', ['rainy', 'sunny']), [0.9, 0.1, 0.5, 0.5])

# create the network structure
bbn = Bbn() \
    .add_node(w_curr) \
    .add_node(w_next) \
    .add_edge(Edge(w_curr, w_next, EdgeType.DIRECTED))

# convert the BBN to a join tree
join_tree = InferenceController.apply(bbn)

for node in [w_curr, w_next]:
    potential = join_tree.get_bbn_potential(node)
    print(node)
    print(potential)
    print('-' * 15)<jupyter_output>0|weather|rainy,sunny
0=rainy|1.00000
0=sunny|0.00000
---------------
2|weather_next|rainy,sunny
2=rainy|0.90000
2=sunny|0.10000
---------------<jupyter_text>We can perform inference on the DBN and use the probability vector of the next state as input to the following time slice.<jupyter_code>import pandas as pd

def get_probs(jt, node):
    entries = jt.get_bbn_potential(node).entries
    return np.array([e.value for e in entries])

P_curr = get_probs(join_tree, w_curr)
jt = InferenceController.reapply(join_tree, {w_curr.id: list(P_curr)})

trace_df = [list(P_curr) + [np.nan]]

for _ in range(20):
    P_next = get_probs(jt, w_next)
    d = np.linalg.norm(P_next - P_curr, 1)
    
    if d < 0.0001:
        break
    
    trace_df.append(list(P_next) + [d])
    P_curr = P_next
            
    jt = InferenceController.reapply(jt, {w_curr.id: list(P_curr)})
    
trace_df = pd.DataFrame(trace_df, columns=['sunny', 'rainy', 'delta'])
P_next
trace_df<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1326478149100257, 'fraction_numerical': 0.021336760925449873, 'mean_word_length': 4.381742738589212, 'pattern_counts': {'":': 0, '<': 26, '<?xml version=': 0, '>': 24, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '459562', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 1508, 'n_tokens_neox': 1418, 'n_words': 378, 'path': 'sphinx/datascience/source/dbn-markov-chain.ipynb', 'repo_name': 'oneoffcoder/books', 'size': '3890'}