<jupyter_start><jupyter_text>Implementing discrete HMMs in Numpy We start with a simple numpy implementation.<jupyter_code>print(42)


# Install necessary libraries

try:
    import jax
except:
    # For cuda version, see https://github.com/google/jax#installation
    %pip install --upgrade "jax[cpu]" 
    import jax

try:
    import jsl
except:
    %pip install git+https://github.com/probml/jsl
    import jsl

try:
    import rich
except:
    %pip install rich
    import rich
import abc
from dataclasses import dataclass
import functools
import itertools

from typing import Any, Callable, NamedTuple, Optional, Union, Tuple


import jax
import jax.numpy as jnp
import matplotlib.pyplot as plt
import numpy as np

import inspect
import inspect as py_inspect

from rich import inspect as r_inspect
from rich import print as r_print

def print_source(fname):
    r_print(py_inspect.getsource(fname))
import jsl
import jsl.hmm.hmm_numpy_lib as hmm_lib_np
#import jsl.hmm.hmm_lib as hmm_lib_jax

normalize = hmm_lib_np.normalize_numpy
print_source(normalize)
#print_source(hmm_lib_np.normalize_numpy)<jupyter_output><empty_output><jupyter_text>Here are some handy utility functions we have already defined.<jupyter_code>normalize = hmm_lib_np.normalize_numpy
print_source(normalize)
#print_source(hmm_lib_np.normalize_numpy)<jupyter_output><empty_output><jupyter_text>We first create the "Ocassionally dishonest casino" model from {cite}`Durbin98`.```{figure} /figures/casino.png:scale: 50%:name: casinoIllustration of the casino HMM.```<jupyter_code># state transition matrix
A = np.array([
    [0.95, 0.05],
    [0.10, 0.90]
])

# observation matrix
B = np.array([
    [1/6, 1/6, 1/6, 1/6, 1/6, 1/6], # fair die
    [1/10, 1/10, 1/10, 1/10, 1/10, 5/10] # loaded die
])

pi = np.array([1, 1]) / 2

(nstates, nobs) = jnp.shape(B)
for i in range(nstates):
    A[i,:] = normalize(A[i,:])[0]
    B[i,:] = normalize(B[i,:])[0]<jupyter_output><empty_output><jupyter_text>Let's bundle the parameters into a structure.<jupyter_code>class HMMNumpy(NamedTuple):
    trans_mat: np.array  # A : (n_states, n_states)
    obs_mat: np.array  # B : (n_states, n_obs)
    init_dist: np.array  # pi : (n_states)
        

params_numpy = HMMNumpy(A, B, pi)
print(params_numpy)<jupyter_output>HMMNumpy(trans_mat=array([[0.95, 0.05],
       [0.1 , 0.9 ]]), obs_mat=array([[0.16666667, 0.16666667, 0.16666667, 0.16666667, 0.16666667,
        0.16666667],
       [0.1       , 0.1       , 0.1       , 0.1       , 0.1       ,
        0.5       ]]), init_dist=array([0.5, 0.5]))<jupyter_text>Function to sample a single sequence of hidden states and discrete observations.<jupyter_code>hmm_sample = hmm_lib_np.hmm_sample_numpy
print_source(hmm_sample)<jupyter_output><empty_output><jupyter_text>Let's sample from this model.<jupyter_code>seq_len = 20
state_seq, obs_seq = hmm_sample(params_numpy, seq_len, random_state=0)
print(state_seq)
print(obs_seq)<jupyter_output>[1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 1 1 0 0 0]
[5 5 5 5 3 5 5 0 4 5 5 5 5 5 4 5 5 3 3 4]
{'fraction_non_alphanumeric': 0.11720039617035326, 'fraction_numerical': 0.05810498514361175, 'mean_word_length': 4.224137931034483, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '181303', 'license': 'MIT', 'n_tokens_mistral': 1312, 'n_tokens_neox': 1163, 'n_words': 295, 'path': '_build/jupyter_execute/chapters/hmm/hmm_discrete_numpy.ipynb', 'repo_name': 'ssm-jax/ssm-book', 'size': '3029'}