<jupyter_start><jupyter_text>$$ $$$$ $$$$$$Hidden Markov Models (HMM)This notebook aims to present a subject widely used in machine learning: Hidden Markov Models or HMM for short. While scikit-learn used to have a library specifically for their implementation, it has since been deprecated due to documentation and stability issues (the repository still exists and you can clone it using this [link](https://github.com/hmmlearn/hmmlearn)). Nonetheless, we will not be using it. The exercises and examples below should provide you with enough information to understand what a HMM is, what is happening underneath the hood and how parameter calibration is important to get the best results.__Dependencies__: _numpy___Kernel__: Python 3__Author__: <NAME> 3To follow the notebook, it is recommended to read [Chapter A of the draft Speech and Language Processing by <NAME> & <NAME>](https://web.stanford.edu/~jurafsky/slp3/A.pdf) [1]. Some of its paragraphs and images are included in the notebook to help the reader but many algorithms were omitted to save time and space!1. [From Markov chains to HMM](sec1)2. [Hidden Markov Model](sec2)2. [Fundamental Problems Connected to HMM](sec3)3. [Complete HMM](sec4)4. [Defining Parameters and Convergence](sec5)7. [To Finish Off](sec6)6. [Going Further](sec7)8. [References](sec8)  1. From Markov chains to HMM  1.1 Markov chains (MC)Suppose we want to model the weather at a certain location and we can use 3 labels: Hot, Warm and Cold. These represent the 3 possible states that a given day can be in and we'll represent them by $q_i$. Let us now consider the problem of forecasting a given day's weather supposing we have access to historical data.We can try to determine the following probability:$$P(q_i = a \vert q_1 ... q_{i-1})$$Which can be translated into: whats the probability of the system being in the state $a$ (the day being Hot, for example), given that in the previous days a certain weather sequence was recorded. To simplify things, we can introduce what is called the Markov assumption. It states that the probability of the system transitioning to state $q_{t+1}$ depends only upon $q_{t}$, or more formally:$$P(q_i = a \vert q_1 ... q_{i-1}) = P(q_i = a | q_{i-1})$$The image below represents examples of Markov chains: 1.2 Components of a MC1. A set of $N$ states (in our example Hot, Warm, Cold):$$Q = q_1,q_2,...,q_N$$2. The transition probability matrix where $a_{ij}$ represents the probability of state $i$ transitioning to state $j$:$$A =\begin{bmatrix}    a_{11} &  \dots  & a_{1N} \\    \vdots & \ddots & \vdots \\    a_{N1} &  \dots  & a_{NN}\end{bmatrix}$$ 3. Initial probability distribution over the $N$ states:$$\pi = \pi_1,\pi_2,...,\pi_N $$If, for instance $\pi_0 = 0$, this means that the state $q_0$ cannot be an initial state.A Markov chain allows us to compute the probability of having, for example, 5 hot days in a row, provided the A matrix and the $\pi$ vector are a good representation of the weather observed.  2.  Hidden Markov ModelLet us now consider a system where we cannot directly infer the states, but what we can do instead is observe certain events that derive from those states. In speech recognition for instance, we can be interested in tagging words (with tags like: subject, verb, object, etc.). However what we can actually observe are the words themselves, so the tags can be considered hidden states. In similarity, the stock market goes through periodical regimes that have an influence on the volatility and the general direction of stock prices. Even though these regimes are not known directly, we can observe the stock prices and infer the regime sequence. 2.1 Components of a HMM Components of a MC plus some other parametersBesides the $Q$ vector of $N$ states, the $\pi$ vector with the initial probability distribution and the transition probability matrix $A$ defined previously we need to introduce the following additional components: 1. A sequence of $T$ observations:$$O = o_1,o_2,...,o_T$$$$ $$2. A sequence of observation likelihoods (not necessarily probabilities but also called emission probabilities, for more information see the [difference between likelihood and probability)](https://stats.stackexchange.com/questions/2641/what-is-the-difference-between-likelihood-and-probability):$$B = b_i(o_t) $$The latter expresses the likelihood of an observation $o_t$ being generated while the system is in state $i$.In literature you will often find the following notation $\lambda = (A,B, \pi)$ to specify a HMM. AssumptionsA first-order hidden Markov model is based on two assumptions:1. The Markov assumption stated earlier:$$P(q_i = a \vert q_1 ... q_{i-1}) = P(q_i = a | q_{i-1})$$2. Outcome independence, which means that the probability of observing $o_i$ depends only on the state responsible for producing the observation:$$P(o_i|q_q,...q_T, o_1,...o_T) = P(o_i|q_i)$$ Let us create a simple Markov model in Python that generates the $A$ matrix from the states provided.<jupyter_code>import numpy as np

'''
    Markov state
'''
class state:
    def __init__(self, **kwargs):
        # Defines the state's id
        if 'id' in kwargs:
            self.id = kwargs['id']

        # State's probability
        if 'p' in kwargs:
            self.probability = kwargs['p']
        else:
            self.probability = 0.5 # default

        # Transition dictionary
        if 'transition' in kwargs:
            self.transition = kwargs['transitions']
        else:
            self.transition = {}
            
        self.emission={}
    
    # Adds a transition from this state to toState with probability trans_prob
    def addTransition(self, toState, trans_prob):
        self.transition[toState.id] = trans_prob

    # Adds observation/emission while in this state with probability emission_prob
    def addEmission(self, observation, emission_prob):
        self.emission[observation] = emission_prob

'''
    Markov process defined by states, transitions and observations
'''
class model:
    def __init__(self, states, obs_seq):
        self.states = states # list with all possible states
        self.obs_seq = obs_seq # list with all possible observations
        self.n = len(states) # number of states
        self.pi = np.array([s.probability for s in states]) # initial probability vector
        self.seq_len = len(obs_seq) # sequence length
        
        # Build A-matrix (state transition probabilities)
        self.a = []
        for thisState in self.states:
            self.a.append([thisState.transition[thatState.id] for thatState in self.states])
        self.a = np.matrix(self.a).reshape([self.n,self.n])
        
        # Build B-matrix (emission probabilities)
        self.b = []
        for currentState in self.states:
            self.b.append([currentState.emission[obs] for obs in self.obs_seq])
        self.b = np.matrix(self.b).reshape([self.n,self.seq_len])<jupyter_output><empty_output><jupyter_text>Weather exampleTo exemplify this model, imagine that you are a climatologist in the year 2799 studying the history of global warming. You cannot find any records of the weather in Toulouse, for the summer of 2018, but you do find a diary, which is listed how many ice creams were eaten every day that summer by one person. Our goal is to use these observations to estimate the temperature every day. We’ll simplify this by assuming there are only two kinds of days (or two states): Hot ($q_1$) and Cold ($q_2$). So the task is as follows:Given a sequence of observations $O$ (each of them an integer representing thenumber of ice creams eaten on a given day) find the ‘hidden’ sequence$Q$ of weather states ($q_1$ or $q_2$) which caused that person to eat the ice cream. Let us create the HMM represented in the image above using the state and model classes.<jupyter_code>'''
    Defining the states 
'''
q1 = state(id='hot', p=0.8)
q2 = state(id='cold', p=0.2)
q1.addTransition(q1, 0.6)
q1.addTransition(q2, 0.4)
q2.addTransition(q1, 0.5)
q2.addTransition(q2, 0.5)
q1.addEmission('1', 0.2)
q1.addEmission('2', 0.4)
q1.addEmission('3', 0.4)
q2.addEmission('1', 0.5)
q2.addEmission('2', 0.4)
q2.addEmission('3', 0.1)

'''
    Creating the model
'''
mdl = model([q1,q2], ['3', '1', '3'])
print('State transition matrix A:')
print(mdl.a)<jupyter_output>State transition matrix A:
[[0.6 0.4]
 [0.5 0.5]]<jupyter_text>3.   Fundamental problems connected to HMMAn influential tutorial by [Rabiner (1989)](https://www.ece.ucsb.edu/Faculty/Rabiner/ece259/Reprints/tutorial%20on%20hmm%20and%20applications.pdf) [2], introduced the idea that hidden Markov models should be characterizedby three fundamental problems:1. __Likelihood__ - For a given HMM $\lambda = (A,B)$ and an observation sequence $O=o_1,...o_T$, calculate the likelihood $P(O|\lambda)$;2. __Decoding__ - For a given sequence $O$ and model $\lambda$ find the most likely (hidden) state sequence $Q$;3. __Learning__ - Given an observation sequence $O$ and a set of states, learn the parameters $A$ and $B$ that define the HMM.  3.1 Forward algorithmOur first problem is to compute the likelihood of a particular observation sequence. For example, given the ice-cream eating HMM in Fig. A.2, what is the probability of the sequence 3 1 3? For a Markov chain, where the surface observations are the same as the hidden events, we could compute the probability of 3 1 3 just by following the states labeled 3 1 3 and multiplying the probabilities along the arcs. For a hidden Markov model, things are not so simple.The forward algorithm is a kind of dynamic programming algorithm, that is, an algorithm that uses a table to storeintermediate values as it builds up the probability of the observation sequence. The forward algorithm computes the observation probability by summing over the probabilities of all possible hidden state paths that could generate the observation sequence, but it does so efficiently by implicitly folding each of these paths into a single forward trellis. This trellis is represented in the images below:  Your turn!Using the trellis represented in the previous image, create the _\_forward_\_ function that takes in a model object or the matrices $A$, $B$ and the vector $\pi$ and outputs the $\alpha$ matrix. You can find the whole algorithm in [1]. When you are done, feed your function with the data from the ice-cream example and check the output! If you want to skip this step just use the _forward_ function already coded.<jupyter_code>def _forward_(A, B, pi):

    '''
    
        Your _forward_ function goes here:
        1) Initialization        
        2) Recursion
        4) Termination
        
        
    '''
    pass<jupyter_output><empty_output><jupyter_text>Compare your code with the following, and check the results:<jupyter_code>def forward(model, verbose):
    if verbose:
        print('\n>>> Forward algorithm. Calculating...')

    edges = []
        
    # Initialization
    alpha = np.multiply(model.pi, [state.emission[model.obs_seq[0]]
                                   for state in model.states]).reshape([-1,1])

    # Recursion
    for j in range(1,model.seq_len):
        # Generates the matrix with the edge values
        edge_matrix = []
        
        for k in range(model.n):
            row = [model.a[i,k]*model.b[k,j] for i in range(model.n)]
            edge_matrix.append(row)
            
        edge_matrix = np.matrix(edge_matrix).reshape([model.n,model.n])
        edges.append(edge_matrix)

        # Adds the next column of the alpha matrix
        next_col = np.matmul(edge_matrix, alpha[:,j-1].reshape([-1,1]))
        alpha = np.concatenate((alpha, next_col), axis=1)

    # Termination
    obs_probability = np.sum(alpha[:,model.seq_len-1])

    if verbose:
        print('\n.Alpha-matrix:')
        print(alpha)

    return alpha
alpha = forward(mdl, verbose=True)<jupyter_output>>>> Forward algorithm. Calculating...

.Alpha-matrix:
[[0.32     0.0404   0.023496]
 [0.02     0.069    0.005066]]<jupyter_text>3.2 DecodingFor any model, such as an HMM, that contains hidden variables, the task of determining which sequence of variables is the underlying source of some sequence of observations is called the decoding task. In the ice-cream domain, given a sequence of ice-cream observations 3 1 3 and an HMM, the task of the decoder is to find the best hidden weather sequence. The most common decoding algorithms for HMMs is the _Viterbi_ algorithm.  Your turn!Using the trellis represented in the previous image, create the  _\_decoder_\_ function that takes in the model or the matrices A, B and the vector $\pi$ and outputs the most likely state sequence. Once again, you can find the whole Viterbi algorithm in [1]!... Or skip ahead and use the _decoder_ function already coded.<jupyter_code>def _decoder_(A, B, pi):
    
    '''
    
    
    
        Your decoder function goes here:
        
        
        
    '''
    pass<jupyter_output><empty_output><jupyter_text>Compare your state sequence with the one generate from the code below:<jupyter_code>'''
    Viterbi decoder - from a sequence of observations infer the most probable
    hidden state sequence.
'''
def decoder(model, verbose):
    if verbose:
        print('\n>>> Viterbi decoder. Calculating...')

    # First Viterbi matrix column
    v = np.multiply(model.pi, [model.b[k,0] for k in range(model.n)]).reshape([-1,1])


    # Keep track of the index of the most likely state
    index_backtrack = [np.where(v == np.max(v))[0][0]]

    for j in range(1,model.seq_len):
        
        # Generates the matrix with the edge values
        edge_matrix = []
        for k in range(model.n):
            row = [model.a[i,k] * model.b[k,j] for i in range(model.n)]
            edge_matrix.append(row)
            
        edge_matrix = np.matrix(edge_matrix).reshape([model.n,model.n])

        likely_state_index = np.where(edge_matrix[:,index_backtrack[j-1]] ==
                             np.max(edge_matrix[:,index_backtrack[j-1]]))[0][0]

        index_backtrack.append(likely_state_index)

        # Adds the next column of the viterbi matrix
        prev_col = v[:,j-1].reshape([1,-1])
        next_col = np.array([np.max(np.multiply(edge_matrix[i,:], prev_col))
                             for i in range(model.n)]).reshape([-1,1])
        v = np.concatenate((v, next_col), axis=1)

        state_seq = [model.states[i].id for i in index_backtrack]
        
    if verbose:
        print('\n.Viterbi matrix:')
        print(v)
        print('\n.Most likely state sequence:')
        print(state_seq)
decoder(mdl, verbose=True)<jupyter_output>>>> Viterbi decoder. Calculating...

.Viterbi matrix:
[[0.32   0.0384 0.0128]
 [0.02   0.064  0.0032]]

.Most likely state sequence:
['hot', 'cold', 'hot']<jupyter_text>4.  Complete HMM class in Python Now let us put it all together in a model class and include the $fit()$ method which will train the model, updating the matrices $A$ and $B$, taking into account the observation sequence provided. The fit method implements the Baum-Welch (or forward backward) algorithm using the forward function you coded earlier. However, is not provided as an exercise due to its length. It is explained in full detail in [1].<jupyter_code>import numpy as np

class model:
    '''
        Initializes a HMM with an observation sequence: obs_seq
        and a list of states: states
    '''
    def __init__(self, obs_seq, states, **kwargs):   
        self.states = states
        self.n = len(self.states) # number of states
        self.obs_seq = obs_seq # list with all possible observations
        self.pi = np.array([s.probability for s in self.states]) # initial probability vector
        self.seq_len = len(obs_seq)
        
        if 'n_symbols' in kwargs:
            self.n_symbols = kwargs['n_symbols']
            
        # Build A-matrix
        self.a = []
        for thisState in self.states:
            self.a.append([thisState.transition[thatState.id] 
                           for thatState in self.states])
        self.a = np.matrix(self.a).reshape([self.n,self.n])

        # Build B-matrix
        self.b = []
        for currentState in self.states:
            self.b.append([currentState.emission[obs] for obs in self.obs_seq])
        self.b = np.matrix(self.b).reshape([self.n,self.seq_len])
            
    '''
        Forward algorithm:
        Calculates the probability of a given observation sequence
        Generates the alpha-matrix
    '''
    def forward(self, verbose):
        if verbose:
            print('\n>>> Forward algorithm. Calculating...')

        edges = []
    
        # Initialization
        self.alpha = np.matrix(np.multiply(self.pi, self.b[:,0].reshape([1,-1]).tolist()[0]).reshape([-1,1]))
        
        # Recursion
        for j in range(1,self.seq_len):
            # Generates the matrix with the edge values
            edge_matrix = []
            for k in range(self.n):
                row = [self.a[i,k]*self.b[k,j] for i in range(self.n)]
                edge_matrix.append(row)
            edge_matrix = np.matrix(edge_matrix).reshape([self.n,self.n])
            edges.append(edge_matrix)

            # Adds the next column of the alpha matrix
            next_col = np.matmul(edge_matrix, self.alpha[:,j-1].reshape([-1,1]))
            self.alpha = np.concatenate((self.alpha, next_col), axis=1)

        # Termination
        self.obs_probability = np.sum(self.alpha[:,self.seq_len-1])

        if verbose:
            print('\n.Alpha-matrix:')
            print(self.alpha)

        return self.alpha

    '''
        Viterbi decoder - from a sequence of observations infer the most probable
        hidden state sequence.
    '''
    def decoder(self, verbose):
        if verbose:
            print('\n>>> Viterbi decoder. Calculating...')

        # First Viterbi matrix column
        v = np.matrix(np.multiply(self.pi, self.b[:,0].reshape([1,-1]).tolist()[0]).reshape([-1,1]))


        # Keep track of the index of the most likely state
        index_backtrack = [np.where(v == np.max(v))[0][0]]

        for j in range(1,self.seq_len):
            # Generates the matrix with the edge values
            edge_matrix = []
            for k in range(self.n):
                row = [self.a[i,k] * self.b[k,j] for i in range(self.n)]
                edge_matrix.append(row)
            edge_matrix = np.matrix(edge_matrix).reshape([self.n,self.n])

            likely_state_index = np.where(edge_matrix[:,index_backtrack[j-1]] ==
                                 np.max(edge_matrix[:,index_backtrack[j-1]]))[0][0]

            index_backtrack.append(likely_state_index)

            # Adds the next column of the viterbi matrix
            prev_col = v[:,j-1].reshape([1,-1])
            next_col = np.array([np.max(np.multiply(edge_matrix[i,:], prev_col))
                       for i in range(self.n)]).reshape([-1,1])
            v = np.concatenate((v, next_col), axis=1)

        state_seq = [self.states[i].id for i in index_backtrack]
        if verbose:
            print('\n.Viterbi matrix:')
            print(v)
            print('\n.Most likely state sequence:')
            print(state_seq)

    '''
        Computes the Beta-matrix using the backward algorithm
    '''
    def backward(self, verbose):
        if verbose:
            print('\n>>> Backward probability. Calculating...')

        # Initialization
        self.beta = np.zeros([self.n, self.seq_len])
        self.beta[:,self.seq_len-1] = np.ones(self.n)

        # Recursion
        for j in range(1,self.seq_len):
            for i in range(self.n):
                self.beta[:,self.seq_len-1-j] += self.beta[i,self.seq_len-j] * \
                                                 np.array(self.a[:,i])[0] * \
                                                 self.b[i,self.seq_len-j]
        self.beta = np.matrix(self.beta)

        if verbose:
            print('\n.Beta matrix:')
            print(self.beta)

        return self.beta

    '''
        Train the HMM - computing the state transition and emission probabilities
        based on the forward-backward (Baum-Welch) algorithm
    '''
    def fit(self, verbose, max_iter):
        if verbose:
            print('\n>>> Training HMM...')
        
        for i in range(max_iter):
            self.forward(verbose=False)
            self.backward(verbose=False)

            # Calculate Xi[t,i,j] matrix
            xi = []
            for t in range(self.seq_len-1):
                num_t = np.zeros([self.n,self.n])
                for i in range(self.n):
                    for j in range(self.n):
                        alpha_ti = self.alpha[i,t]
                        a_ij = self.a[i,j]
                        b_jt1 = self.b[j,t+1]
                        beta_jt1 = self.beta[j,t+1]
                        num_t[i,j] = alpha_ti*a_ij*b_jt1*beta_jt1
                xi.append(np.matrix(num_t))
                xi[t] /= np.sum(np.sum(num_t,axis=0))
                
                
            # Calculate gamma matrix
            gamma = [np.sum(xi[i], axis=1) for i in range(len(xi))]
            
            # Calculate a-hat matrix
            a_hat_num = np.sum(xi,axis=0)
            a_hat_den = np.array(np.sum(gamma, axis=0)).reshape([1,-1])[0]
            a_hat = np.matrix(np.zeros([self.n, self.n]))
            for i in range(self.n):
                for j in range(self.n):
                    a_hat[i,j] = a_hat_num[i,j] / a_hat_den[i]
        
            # Calculate probability of symbol k while in a certain state
            symbol_prob = np.matrix(np.zeros([self.n, self.n_symbols]))
            for k in range(1,self.n_symbols+1):
                v_k = str(k)
                for kk in range(self.seq_len-1):
                    if str(self.obs_seq[kk]) == v_k:
                        symbol_prob[:,k-1] += gamma[kk]
            
            # Calculate b-hat matrix
            b_hat_num = np.matrix(np.zeros([self.n, self.seq_len]))
            b_hat = b_hat_num
            for t in range(self.seq_len):
                for k in range(1,self.n_symbols+1):
                    for j in range(self.n):
                        if self.obs_seq[t] == str(k):
                            b_hat_num[j,t] = symbol_prob[j,k-1] / a_hat_den[j]
                        
            # Update matrices
            self.a = a_hat
            self.b = b_hat

            '''
            If you want to see the probability of the sequence 
            converging as the training process goes on
            UNCOMMENT LINE BELOW!
            '''
            print(self.obs_probability)
        
        if verbose:
            print('\n.State transitions probabilities estimates (A-hat matrix):')
            print(self.a)
            print('\n.Emission probabilities estimates (B-hat matrix):')
            print(self.b)
            print('\n.Sequence probability:')
            print(self.obs_probability)<jupyter_output><empty_output><jupyter_text>5.  Defining parameters and verifying convergenceYou can now use the class defined above and its methods to play with the initial parameters ($\pi$, $A$ and $B$) and the observation sequence to see how the results vary. Use the code below as a starting example:<jupyter_code>'''
    Defining the states 
'''
q1 = state(id='hot', p=0.5)
q2 = state(id='cold', p=0.5)
q1.addTransition(q1, 0.7)
q1.addTransition(q2, 0.3)
q2.addTransition(q1, 0.3)
q2.addTransition(q2, 0.7)
q1.addEmission('1', 0.1)
q1.addEmission('2', 0.4)
q1.addEmission('3', 0.5)
q2.addEmission('1', 0.7)
q2.addEmission('2', 0.2)
q2.addEmission('3', 0.1)

'''
    Creating the model
'''
sequence = ['3', '3', '3', '2', '2', '1', '1', '1', '1', '1', '2', '3', '2', '3', '3', '3']
mdl = model(sequence, [q1,q2], n_symbols=3)

print('\n.A-matrix before fit')
print(mdl.a)

print('\n.B-matrix before fit')
print(mdl.b)

# updates A and B matrices using the observation sequence fed to the model
mdl.fit(max_iter=100, verbose=False)

print('\n.A-matrix after fit')
print(mdl.a)

print('\n.B-matrix after fit')
print(mdl.b)

mdl.decoder(verbose=True)<jupyter_output>.A-matrix before fit
[[0.7 0.3]
 [0.3 0.7]]

.B-matrix before fit
[[0.5 0.5 0.5 0.4 0.4 0.1 0.1 0.1 0.1 0.1 0.4 0.5 0.4 0.5 0.5 0.5]
 [0.1 0.1 0.1 0.2 0.2 0.7 0.7 0.7 0.7 0.7 0.2 0.1 0.2 0.1 0.1 0.1]]
9.956492805475383e-08
5.067808196951614e-07
8.825428264116487e-07
8.962858080926481e-07
8.664346068270807e-07
8.597676890479551e-07
8.647896979754734e-07
8.761080386332066e-07
8.916103254399431e-07
9.0982380705969e-07
9.295012810873849e-07
9.496408125663863e-07
9.694853377786605e-07
9.884930949947306e-07
1.006302758305215e-06
1.0226996494718184e-06
1.037583690224978e-06
1.0509399275342271e-06
1.0628128734569957e-06
1.0732855323308252e-06
1.0824632759099286e-06
1.090462119171787e-06
1.097400621905329e-06
1.1033945711606157e-06
1.1085536880097225e-06
1.1129797536056285e-06
1.1167657032623204e-06
1.1199953656890108e-06
1.1227436215084102e-06
1.125076824640217e-06
1.1270533786707377e-06
1.12872439406628e-06
1.1301343756516058e-06
1.1313219063750583e-06
1.1323203051669659e-06
1.13315824509572[...]<jupyter_text>As expected the A and B matrices are different after the fit. And you should see that the initial parameters change these matrices even after the fit is performed. Try it yourself.  Your turn!What happens in the following cases?1. The input observation sequence is too long. Try it with arrays as long as +1000 elements. Uncomment the line __print(self.obs_probability)__ in the _fit_ method of the class _model_ to check if the observation sequence probability converges.2. You change the input parameters. Does the _Viterbi_ algorithm have the same output? What about the observation probability?3. You add more states. Does it always converge?Use the code below as an example and feel free to play around. Plot the observation sequence probability for different initial parameters. Check if it convergences. Ajust the maximum number of iterations, __max_iter__ if need be.<jupyter_code>q1 = state(id='hot', p=0.5) # changed the initial probability
q2 = state(id='cold', p=0.5) # changed the initial probability

# Transitions probabilities are now uniform and all equal to 0.5
q1.addTransition(q1, 0.9)
q1.addTransition(q2, 0.1)
q2.addTransition(q1, 0.3)
q2.addTransition(q2, 0.7)

# Different emission probabilities
q1.addEmission('1', 0.1)
q1.addEmission('2', 0.4)
q1.addEmission('3', 0.5)
q2.addEmission('1', 0.7)
q2.addEmission('2', 0.2)
q2.addEmission('3', 0.1)

sequence = ['1', '1', '3', '3', '2', '3', '2', '1', '2', '2', '1'] # Try long sequences
mdl = model(sequence, [q1,q2], n_symbols=3)
mdl.fit(max_iter=100, verbose=True) # Use verbose=True to see the calculations
mdl.decoder(verbose=True)
'''


    Your own sandbox
    Use the model and state classes to create your own HMM
    Test it!
    
    
'''<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10838316680904803, 'fraction_numerical': 0.04412584035954389, 'mean_word_length': 2.7926468516692493, 'pattern_counts': {'":': 0, '<': 36, '<?xml version=': 0, '>': 60, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '541009', 'license': 'MIT', 'n_tokens_mistral': 8951, 'n_tokens_neox': 8224, 'n_words': 2786, 'path': 'hmm.ipynb', 'repo_name': 'gabmoreira/datascience', 'size': '26950'}