<jupyter_start><jupyter_text>Sometimes looking at the learned coefficients of a neural network can provide insight into the learning behavior. For example if weights look unstructured, maybe some were not used at all, or if very large coefficients exist, maybe regularization was too low or the learning rate too high.This example shows how to plot some of the first layer weights in a MLPClassifier trained on the MNIST dataset.The input data consists of 28x28 pixel handwritten digits, leading to 784 features in the dataset. Therefore the first layer weight matrix have the shape (784, hidden_layer_sizes[0]). We can therefore visualize a single column of the weight matrix as a 28x28 pixel image.To make the example run faster, we use very few hidden units, and train only for a very short time. Training longer would result in weights with a much smoother spatial appearance.  New to Plotly?Plotly's Python library is free and open source! [Get started](https://plot.ly/python/getting-started/) by downloading the client and [reading the primer](https://plot.ly/python/getting-started/).You can set up Plotly to work in [online](https://plot.ly/python/getting-started/initialization-for-online-plotting) or [offline](https://plot.ly/python/getting-started/initialization-for-offline-plotting) mode, or in [jupyter notebooks](https://plot.ly/python/getting-started/start-plotting-online).We also have a quick-reference [cheatsheet](https://images.plot.ly/plotly-documentation/images/python_cheat_sheet.pdf) (new!) to help you get started!  Version<jupyter_code>import sklearn
sklearn.__version__<jupyter_output><empty_output><jupyter_text>Imports This tutorial imports [fetch_mldata](http://scikit-learn.org/stable/modules/generated/sklearn.datasets.fetch_mldata.htmlsklearn.datasets.fetch_mldata) and [MLPClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.htmlsklearn.neural_network.MLPClassifier).<jupyter_code>print(__doc__)

import plotly.plotly as py
import plotly.graph_objs as go
from plotly import tools

import matplotlib.pyplot as plt
import numpy as np
from sklearn.datasets import fetch_mldata
from sklearn.neural_network import MLPClassifier<jupyter_output>Automatically created module for IPython interactive environment<jupyter_text>Calculations<jupyter_code>mnist = fetch_mldata("MNIST original")
# rescale the data, use the traditional train/test split
X, y = mnist.data / 255., mnist.target
X_train, X_test = X[:60000], X[60000:]
y_train, y_test = y[:60000], y[60000:]

# mlp = MLPClassifier(hidden_layer_sizes=(100, 100), max_iter=400, alpha=1e-4,
#                     solver='sgd', verbose=10, tol=1e-4, random_state=1)
mlp = MLPClassifier(hidden_layer_sizes=(50,), max_iter=10, alpha=1e-4,
                    solver='sgd', verbose=10, tol=1e-4, random_state=1,
                    learning_rate_init=.1)

mlp.fit(X_train, y_train)
print("Training set score: %f" % mlp.score(X_train, y_train))
print("Test set score: %f" % mlp.score(X_test, y_test))<jupyter_output>Iteration 1, loss = 0.32212731
Iteration 2, loss = 0.15738787
Iteration 3, loss = 0.11647274
Iteration 4, loss = 0.09631113
Iteration 5, loss = 0.08074513
Iteration 6, loss = 0.07163224
Iteration 7, loss = 0.06351392
Iteration 8, loss = 0.05694146
Iteration 9, loss = 0.05213487
Iteration 10, loss = 0.04708320
Training set score: 0.985733
Test set score: 0.971000<jupyter_text>Plot Results<jupyter_code>fig = tools.make_subplots(rows=4, cols=4,
                          print_grid=False)

def matplotlib_to_plotly(cmap, pl_entries):
    h = 1.0/(pl_entries-1)
    pl_colorscale = []
    
    for k in range(pl_entries):
        C = map(np.uint8, np.array(cmap(k*h)[:3])*255)
        pl_colorscale.append([k*h, 'rgb'+str((C[0], C[1], C[2]))])
        
    return pl_colorscale
cmap = matplotlib_to_plotly(plt.cm.gray, 5)
# use global min / max to ensure all weights are shown on the same scale
vmin, vmax = mlp.coefs_[0].min(), mlp.coefs_[0].max()

order = np.linspace(0, 15, 16)

for coef, i in zip(mlp.coefs_[0].T, order):
    p1 = go.Heatmap(z=coef.reshape(28, 28), 
                    colorscale=cmap, showscale=False)
    fig.append_trace(p1, int(i/4+1), int(i%4+1))
    
for i in map(str,range(1, 17)):
        y = 'yaxis'+ i
        x = 'xaxis' + i
        fig['layout'][y].update(autorange='reversed',
                                showticklabels=False, ticks='')
        fig['layout'][x].update(showticklabels=False, ticks='')
        
fig['layout'].update(height=700)
py.iplot(fig)
from IPython.display import display, HTML

display(HTML('<link href="//fonts.googleapis.com/css?family=Open+Sans:600,400,300,200|Inconsolata|Ubuntu+Mono:400,700" rel="stylesheet" type="text/css" />'))
display(HTML('<link rel="stylesheet" type="text/css" href="http://help.plot.ly/documentation/all_static/css/ipython-notebook-custom.css">'))

! pip install git+https://github.com/plotly/publisher.git --upgrade
import publisher
publisher.publish(
    'Visualization of MLP Weights on MNIST .ipynb', 'scikit-learn/plot-mnist-filters/', 'Visualization of MLP Weights on MNIST  | plotly',
    ' ',
    title = 'Visualization of MLP Weights on MNIST  | plotly',
    name = 'Visualization of MLP Weights on MNIST ',
    has_thumbnail='true', thumbnail='thumbnail/mnist-filters.jpg', 
    language='scikit-learn', page_type='example_index',
    display_as='neural_networks', order=1,
    ipynb= '~Diksha_Gabha/3490')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11983924004384362, 'fraction_numerical': 0.04420898794300329, 'mean_word_length': 4.564024390243903, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '134176', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 2033, 'n_tokens_neox': 1833, 'n_words': 468, 'path': '_posts/scikit/visualizaion-of-mlp-on-mnist/Visualization of MLP Weights on MNIST .ipynb', 'repo_name': 'bmb804/documentation', 'size': '5474'}