<jupyter_start><jupyter_text>MNIST Softmax Estimation Note: This notebook is desinged to run with Python3 and CPU (no GPU) runtime.![Python 3 and CPU runtime](https://raw.githubusercontent.com/enakai00/colab_tfbook/master/docs/imgs/runtime_cpu.png) This notebook uses TensorFlow2.x.<jupyter_code>%tensorflow_version 2.x<jupyter_output>TensorFlow 2.x selected.<jupyter_text>[MSE-01]Import modules.<jupyter_code>import numpy as np
from pandas import DataFrame
import matplotlib.pyplot as plt

import tensorflow as tf
from tensorflow.keras import layers, models
from tensorflow.keras.datasets import mnist<jupyter_output><empty_output><jupyter_text>[MSE-02]Download the MNIST dataset and store into NumPy arrays.<jupyter_code>(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
train_images = train_images.reshape(
                  (len(train_images), 784)).astype('float32') / 255
test_images = test_images.reshape(
                  (len(test_images), 784)).astype('float32') / 255
train_labels = tf.keras.utils.to_categorical(train_labels, 10)
test_labels = tf.keras.utils.to_categorical(test_labels, 10)<jupyter_output>Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz
11493376/11490434 [==============================] - 0s 0us/step<jupyter_text>[MSE-03]Define a model for the softmax estimation.<jupyter_code>model = models.Sequential()
model.add(layers.Dense(10, activation='softmax', input_shape=(28*28,),
                       name='softmax'))

model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
softmax (Dense)              (None, 10)                7850      
=================================================================
Total params: 7,850
Trainable params: 7,850
Non-trainable params: 0
_________________________________________________________________<jupyter_text>[MSE-04]Compile the model using the Adam optimizer, and Cross entroy as a loss function.<jupyter_code>model.compile(optimizer='adam', 
              loss='categorical_crossentropy',
              metrics=['acc'])<jupyter_output><empty_output><jupyter_text>[MSE-05]Train the model. It acheves the 92.7% accuracy for the test dataset.<jupyter_code>history = model.fit(train_images, train_labels,
                    validation_data=(test_images, test_labels),
                    batch_size=128, epochs=10)<jupyter_output>Train on 60000 samples, validate on 10000 samples
Epoch 1/10
60000/60000 [==============================] - 2s 38us/sample - loss: 0.6809 - acc: 0.8316 - val_loss: 0.3845 - val_acc: 0.9000
Epoch 2/10
60000/60000 [==============================] - 1s 20us/sample - loss: 0.3629 - acc: 0.9025 - val_loss: 0.3195 - val_acc: 0.9129
Epoch 3/10
60000/60000 [==============================] - 1s 21us/sample - loss: 0.3200 - acc: 0.9120 - val_loss: 0.2975 - val_acc: 0.9148
Epoch 4/10
60000/60000 [==============================] - 1s 21us/sample - loss: 0.2997 - acc: 0.9169 - val_loss: 0.2884 - val_acc: 0.9194
Epoch 5/10
60000/60000 [==============================] - 1s 21us/sample - loss: 0.2877 - acc: 0.9196 - val_loss: 0.2770 - val_acc: 0.9224
Epoch 6/10
60000/60000 [==============================] - 1s 21us/sample - loss: 0.2796 - acc: 0.9216 - val_loss: 0.2751 - val_acc: 0.9250
Epoch 7/10
60000/60000 [==============================] - 1s 20us/sample - loss: 0.2737 - acc: 0.9234 - val_loss: 0.[...]<jupyter_text>[MSE-06]Show examples of the prediction result. Three for correct preditions and three for incorrect predictions.<jupyter_code>p_val = model.predict(np.array(test_images))
df = DataFrame({'pred': list(map(np.argmax, p_val)),
                'label': list(map(np.argmax, test_labels))})
correct = df[df['pred']==df['label']]
incorrect = df[df['pred']!=df['label']]

fig = plt.figure(figsize=(8, 15))
for i in range(10):
    indices = list(correct[correct['pred']==i].index[:3]) \
                + list(incorrect[incorrect['pred']==i].index[:3])
    for c, image in enumerate(test_images[indices]):
        subplot = fig.add_subplot(10, 6, i*6+c+1)
        subplot.set_xticks([])
        subplot.set_yticks([])
        subplot.set_title('%d / %d' % (i, df['label'][indices[c]]))
        subplot.imshow(image.reshape((28, 28)),
                       vmin=0, vmax=1, cmap=plt.cm.gray_r)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.2035103310375472, 'fraction_numerical': 0.07931570762052877, 'mean_word_length': 4.346793349168646, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '55715', 'license': 'Apache-2.0', 'n_tokens_mistral': 1740, 'n_tokens_neox': 1458, 'n_words': 304, 'path': 'Chapter02/4. MNIST softmax estimation.ipynb', 'repo_name': 'enakai00/jupyter_tfbook2', 'size': '4501'}