<jupyter_start><jupyter_text>Multilayer Perceptron Classifier Let's set some setting for this Jupyter Notebook.<jupyter_code>%matplotlib inline   
from warnings import filterwarnings
filterwarnings("ignore")
import os
os.environ['MKL_THREADING_LAYER'] = 'GNU'
os.environ['THEANO_FLAGS'] = 'device=cpu'    
    
import numpy as np
import pandas as pd
import pymc3 as pm
import seaborn as sns
import matplotlib.pyplot as plt
np.random.seed(12345)
rc = {'xtick.labelsize': 20, 'ytick.labelsize': 20, 'axes.labelsize': 20, 'font.size': 20, 
      'legend.fontsize': 12.0, 'axes.titlesize': 10, "figure.figsize": [12, 6]}
sns.set(rc = rc)
from IPython.core.interactiveshell import InteractiveShell
InteractiveShell.ast_node_interactivity = "all"<jupyter_output><empty_output><jupyter_text>Now, let's import the `MLPClassifier` algorithm from the `pymc-learn` package.<jupyter_code>import pmlearn
from pmlearn.neural_network import MLPClassifier
print('Running on pymc-learn v{}'.format(pmlearn.__version__))<jupyter_output>Running on pymc-learn v0.0.1.rc0<jupyter_text>Step 1: Prepare the dataGenerate synthetic data.<jupyter_code>from sklearn.datasets import make_moons
from sklearn.preprocessing import scale
import theano
floatX = theano.config.floatX

X, y = make_moons(noise=0.2, random_state=0, n_samples=1000)
X = scale(X)
X = X.astype(floatX)
y = y.astype(floatX)

## Plot the data 
fig, ax = plt.subplots(figsize=(12, 8))
ax.scatter(X[y==0, 0], X[y==0, 1], label='Class 0')
ax.scatter(X[y==1, 0], X[y==1, 1], color='r', label='Class 1')
sns.despine(); ax.legend()
ax.set(xlabel='X', ylabel='y', title='Toy binary classification data set');
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)<jupyter_output><empty_output><jupyter_text>Step 2: Instantiate a model<jupyter_code>model = MLPClassifier()<jupyter_output><empty_output><jupyter_text>Step 3: Perform Inference<jupyter_code>model.fit(X_train, y_train)<jupyter_output>Average Loss = 140.51: 100%|██████████| 200000/200000 [02:46<00:00, 1203.69it/s]
Finished [100%]: Average Loss = 140.52<jupyter_text>Step 4: Diagnose convergence<jupyter_code>model.plot_elbo()
pm.traceplot(model.trace);
pm.forestplot(model.trace, varnames=["w_in_1"]);<jupyter_output><empty_output><jupyter_text>Step 5: Critize the model<jupyter_code>pm.summary(model.trace)
pm.plot_posterior(model.trace, varnames=["w_in_1"], 
                 figsize = [14, 8]);<jupyter_output><empty_output><jupyter_text>Step 6: Use the model for prediction<jupyter_code>y_pred = model.predict(X_test)
y_pred
model.score(X_test, y_test)
model.save('pickle_jar/mlpc')<jupyter_output><empty_output><jupyter_text>Use already trained model for prediction<jupyter_code>model_new = MLPClassifier()
model_new.load('pickle_jar/mlpc')
model_new.score(X_test, y_test)<jupyter_output>100%|██████████| 2000/2000 [00:00<00:00, 2619.50it/s]<jupyter_text>MCMC  Perform inference<jupyter_code>model2 = MLPClassifier()
model2.fit(X_train, y_train, inference_type='nuts')<jupyter_output>Multiprocess sampling (4 chains in 4 jobs)
NUTS: [w_2_out, w_1_2, w_in_1]
100%|██████████| 2500/2500 [04:25<00:00,  9.42it/s]
There were 125 divergences after tuning. Increase `target_accept` or reparameterize.
There were 228 divergences after tuning. Increase `target_accept` or reparameterize.
There were 210 divergences after tuning. Increase `target_accept` or reparameterize.
There were 32 divergences after tuning. Increase `target_accept` or reparameterize.
The estimated number of effective samples is smaller than 200 for some parameters.<jupyter_text>Diagnose convergence<jupyter_code>pm.traceplot(model2.trace, varnames=["w_in_1"]);
pm.gelman_rubin(model2.trace)
pm.energyplot(model2.trace);
pm.forestplot(model2.trace, varnames=["w_in_1"]);<jupyter_output><empty_output><jupyter_text>Criticize the model<jupyter_code>pm.summary(model2.trace)
pm.plot_posterior(model2.trace, varnames=["w_in_1"], 
                 figsize = [14, 8]);<jupyter_output><empty_output><jupyter_text>Use the model for prediction<jupyter_code>y_pred2 = model2.predict(X_test)
y_pred2
model2.score(X_test, y_test)
model2.save('pickle_jar/mlpc2')
model2_new = MLPClassifier()
model2_new.load('pickle_jar/mlpc2')
model2_new.score(X_test, y_test)<jupyter_output>100%|██████████| 2000/2000 [00:00<00:00, 2550.55it/s]
{'fraction_non_alphanumeric': 0.1377094330961671, 'fraction_numerical': 0.0498049116364471, 'mean_word_length': 7.629702970297029, 'pattern_counts': {'":': 1, '<': 52, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '539975', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1853, 'n_tokens_neox': 1663, 'n_words': 318, 'path': 'docs/notebooks/MLPClassifier.ipynb', 'repo_name': 'john-veillette/pymc-learn', 'size': '4357'}