<jupyter_start><jupyter_text>High Accuracy CNN for MNIST  Build your own CNN from scratch and try to achieve the highest possible accuracy on MNIST.<jupyter_code># Exercise: Build your own CNN from scratch and try to achieve the highest possible accuracy on MNIST.
# he following model uses 2 convolutional layers, followed by 1 pooling layer, then dropout 25%, 
# then a dense layer, another dropout layer but with 50% dropout, and finally the output layer. 
# It reaches about 99.2% accuracy on the test set. This places this model roughly in the top 20% 
# in the MNIST Kaggle competition (if we ignore the models with an accuracy greater than 99.79% 
# which were most likely trained on the test set, as explained by <NAME> in this post). 
# Can you do better? To reach 99.5 to 99.7% accuracy on the test set, you need to add image augmentation, 
# batch norm, use a learning schedule such as 1-cycle, and possibly create an ensemble.
# Python ≥3.5 is required
import sys
assert sys.version_info >= (3, 5)

# Is this notebook running on Colab or Kaggle?
IS_COLAB = "google.colab" in sys.modules
IS_KAGGLE = "kaggle_secrets" in sys.modules

# Scikit-Learn ≥0.20 is required
import sklearn
assert sklearn.__version__ >= "0.20"

# TensorFlow ≥2.0 is required
import tensorflow as tf
from tensorflow import keras
assert tf.__version__ >= "2.0"

if not tf.config.list_physical_devices('GPU'):
    print("No GPU was detected. CNNs can be very slow without a GPU.")
    if IS_COLAB:
        print("Go to Runtime > Change runtime and select a GPU hardware accelerator.")
    if IS_KAGGLE:
        print("Go to Settings > Accelerator and select GPU.")

# Common imports
import numpy as np
import os

# to make this notebook's output stable across runs
np.random.seed(42)
tf.random.set_seed(42)

# To plot pretty figures
%matplotlib inline
import matplotlib as mpl
import matplotlib.pyplot as plt
mpl.rc('axes', labelsize=14)
mpl.rc('xtick', labelsize=12)
mpl.rc('ytick', labelsize=12)

# Where to save the figures
PROJECT_ROOT_DIR = "."
CHAPTER_ID = "cnn"
IMAGES_PATH = os.path.join(PROJECT_ROOT_DIR, "images", CHAPTER_ID)
os.makedirs(IMAGES_PATH, exist_ok=True)

def save_fig(fig_id, tight_layout=True, fig_extension="png", resolution=300):
    path = os.path.join(IMAGES_PATH, fig_id + "." + fig_extension)
    print("Saving figure", fig_id)
    if tight_layout:
        plt.tight_layout()
(X_train_full, y_train_full), (X_test, y_test) = keras.datasets.mnist.load_data()
X_train_full = X_train_full / 255.
X_test = X_test / 255.
X_train, X_valid = X_train_full[:-5000], X_train_full[-5000:]
y_train, y_valid = y_train_full[:-5000], y_train_full[-5000:]

X_train = X_train[..., np.newaxis]
X_valid = X_valid[..., np.newaxis]
X_test = X_test[..., np.newaxis]
keras.backend.clear_session()
tf.random.set_seed(42)
np.random.seed(42)

model = keras.models.Sequential([
    keras.layers.Conv2D(32, kernel_size=3, padding="same", activation="relu"),
    keras.layers.Conv2D(64, kernel_size=3, padding="same", activation="relu"),
    keras.layers.MaxPool2D(),
    keras.layers.Flatten(),
    keras.layers.Dropout(0.25),
    keras.layers.Dense(128, activation="relu"),
    keras.layers.Dropout(0.5),
    keras.layers.Dense(10, activation="softmax")
])
model.compile(loss="sparse_categorical_crossentropy", optimizer="nadam",
              metrics=["accuracy"])

model.fit(X_train, y_train, epochs=10, validation_data=(X_valid, y_valid))
model.evaluate(X_test, y_test)<jupyter_output>Epoch 1/10
1719/1719 [==============================] - 70s 40ms/step - loss: 0.3724 - accuracy: 0.8834 - val_loss: 0.0457 - val_accuracy: 0.9886
Epoch 2/10
1719/1719 [==============================] - 67s 39ms/step - loss: 0.0795 - accuracy: 0.9754 - val_loss: 0.0424 - val_accuracy: 0.9878
Epoch 3/10
1719/1719 [==============================] - 70s 41ms/step - loss: 0.0583 - accuracy: 0.9814 - val_loss: 0.0350 - val_accuracy: 0.9902
Epoch 4/10
1719/1719 [==============================] - 68s 40ms/step - loss: 0.0519 - accuracy: 0.9837 - val_loss: 0.0335 - val_accuracy: 0.9912
Epoch 5/10
1719/1719 [==============================] - 70s 41ms/step - loss: 0.0402 - accuracy: 0.9867 - val_loss: 0.0353 - val_accuracy: 0.9904
Epoch 6/10
1719/1719 [==============================] - 66s 38ms/step - loss: 0.0351 - accuracy: 0.9892 - val_loss: 0.0398 - val_accuracy: 0.9916
Epoch 7/10
1719/1719 [==============================] - 65s 38ms/step - loss: 0.0320 - accuracy: 0.9897 - val_loss: 0.0356 -[...]
{'fraction_non_alphanumeric': 0.16216216216216217, 'fraction_numerical': 0.07505025686843869, 'mean_word_length': 4.369304556354916, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '4508', 'license': 'Apache-2.0', 'n_tokens_mistral': 1793, 'n_tokens_neox': 1538, 'n_words': 454, 'path': 'chapter14/high_accuracy_MNIST.ipynb', 'repo_name': 'sfefilatyev/tensorflow_excercises', 'size': '4483'}