<jupyter_start><jupyter_text>Effect of the variance - Overlapping<jupyter_code>#Class with small variances, little overlapping possible
n_c1 = 70
P1 = generate_transtion_matrix(3) #random transition matrix with 3 states
params1 = {0: [-2, 0.5], 1 : [0, 0.5], 2: [2, 0.5]} #state : [mu, sigma]
len_range1 = [50, 100]
Y1 = generate_samples(n_c1, len_range1, P1, params1, noise=0.1)


#class with larger variance, possible large overlapping
n_c2 = 70
params2 = {0: [-1, 2], 1 : [3, 1.5], 2: [1, 1.5]} #state : [mu, sigma]
len_range2 = [50, 100]
Y2 = generate_samples(n_c2, len_range2, P1, params2, noise=0.1)
P1
plt.figure(figsize=(8, 7))
sns.distplot(np.concatenate(Y1), hist=False, label="Class 1 with small variances")
sns.distplot(np.concatenate(Y2), hist=False, label="Class 1 with large variances")
plt.show()<jupyter_output><empty_output><jupyter_text>We can see that, in the case of low variances, one can easily discriminate between the Gaussians components of the mixture whereas when the variance is large, it becomes impossible. We see below the effect on the estimators of the parameters.<jupyter_code>X_data = {1:Y1, 2:Y2}
%time models = build_hmm_models(X_data, 2, 3, bayesian_hmm, n_iter_gibbs=2000, max_obs = 900)
models[0].monitor_convergence()
models[1].monitor_convergence()<jupyter_output><empty_output><jupyter_text>Using it for classification We modify a bit the previous specifications in order to make the classication task more challenging and add a small noise to the sequences.<jupyter_code>#Class with small variances, little overlapping possible
n_c1 = 70
P1 = generate_transtion_matrix(3) #random transition matrix with 3 states
params1 = {0: [-2, 1], 1 : [0, 0.5], 2: [2, 0.5]} #state : [mu, sigma]
len_range1 = [50, 100]
Y1 = generate_samples(n_c1, len_range1, P1, params1, noise=0.1)


#class with larger variance, possible large overlapping
n_c2 = 70
params2 = {0: [-2, 1.5], 1 : [3, 1.5], 2: [1, 1.5]} #state : [mu, sigma]
len_range2 = [50, 100]
P2 = generate_transtion_matrix(3)
Y2 = generate_samples(n_c2, len_range2, P2, params2, noise=0.1)
plt.figure(figsize=(8, 7))
sns.distplot(np.concatenate(Y1), hist=False, label="Class 1")
sns.distplot(np.concatenate(Y2), hist=False, label="Class 2")
plt.show()
X_train = {1:Y1[:10], 2:Y2[:10]}
X_test = Y1[10:] + Y2[10:]
y_test = [1] * len(Y1[10:]) + [2] * len(Y2[10:])
%time models = build_hmm_models(X_train, 2, 3, bayesian_hmm, n_iter_gibbs=2000, max_obs = 900)
preds = assign_classes(X_test, models)
print("Accuracy : ", np.sum(np.array(y_test) == np.array(preds)) / len(preds))
sns.heatmap(pd.DataFrame(confusion_matrix(y_test, preds)), annot=True, cbar=False)
plt.title("Confusion Matrix")
plt.show()
X_train
type(X_test)
preds<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14629697190806276, 'fraction_numerical': 0.06603429405326523, 'mean_word_length': 6.140625, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '615279', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1161, 'n_tokens_neox': 1051, 'n_words': 291, 'path': 'Bayesian_hmm/Toy example - Bayesian HMM.ipynb', 'repo_name': 'mehdibnc/Bayesian_Time_Series_Classification', 'size': '2741'}