<jupyter_start><jupyter_text>Deep Neural Network in TensorFlow In this notebook, we improve on our [intermediate neural net](https://github.com/jonkrohn/DLTFpT/blob/master/notebooks/intermediate_net_in_tensorflow.ipynb) by incorporating the theory we've covered since. [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/jonkrohn/DLTFpT/blob/master/notebooks/deep_net_in_tensorflow.ipynb)  Load dependencies<jupyter_code>import tensorflow
from tensorflow.keras.datasets import mnist
from tensorflow.keras.utils import to_categorical
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Dropout # new!
from tensorflow.keras.layers import BatchNormalization # new!
from tensorflow.keras.optimizers import SGD
from matplotlib import pyplot as plt<jupyter_output><empty_output><jupyter_text>Load data<jupyter_code>(X_train, y_train), (X_valid, y_valid) = mnist.load_data()<jupyter_output><empty_output><jupyter_text>Preprocess data<jupyter_code>X_train = X_train.reshape(60000, 784).astype('float32')
X_valid = X_valid.reshape(10000, 784).astype('float32')
X_train /= 255
X_valid /= 255
n_classes = 10
y_train = to_categorical(y_train, n_classes)
y_valid = to_categorical(y_valid, n_classes)<jupyter_output><empty_output><jupyter_text>Design neural network architecture<jupyter_code>model = Sequential()

model.add(Dense(64, activation='relu', input_shape=(784,)))
model.add(BatchNormalization())

model.add(Dense(64, activation='relu'))
model.add(BatchNormalization())

model.add(Dense(64, activation='relu'))
model.add(BatchNormalization())
model.add(Dropout(0.2))

model.add(Dense(10, activation='softmax'))
model.summary()<jupyter_output>Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 64)                50240     
_________________________________________________________________
batch_normalization (BatchNo (None, 64)                256       
_________________________________________________________________
dense_1 (Dense)              (None, 64)                4160      
_________________________________________________________________
batch_normalization_1 (Batch (None, 64)                256       
_________________________________________________________________
dense_2 (Dense)              (None, 64)                4160      
_________________________________________________________________
batch_normalization_2 (Batch (None, 64)                256       
________________________________________________________[...]<jupyter_text>Configure model<jupyter_code>model.compile(loss='categorical_crossentropy', optimizer='nadam', metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Train!<jupyter_code>model.fit(X_train, y_train, batch_size=128, epochs=20, verbose=1, validation_data=(X_valid, y_valid))<jupyter_output>Train on 60000 samples, validate on 10000 samples
Epoch 1/20
60000/60000 [==============================] - 3s 45us/sample - loss: 0.3792 - accuracy: 0.8857 - val_loss: 0.1684 - val_accuracy: 0.9479
Epoch 2/20
60000/60000 [==============================] - 2s 31us/sample - loss: 0.1519 - accuracy: 0.9542 - val_loss: 0.1251 - val_accuracy: 0.9596
Epoch 3/20
60000/60000 [==============================] - 2s 26us/sample - loss: 0.1110 - accuracy: 0.9663 - val_loss: 0.1087 - val_accuracy: 0.9669
Epoch 4/20
60000/60000 [==============================] - 2s 30us/sample - loss: 0.0885 - accuracy: 0.9726 - val_loss: 0.0986 - val_accuracy: 0.9711
Epoch 5/20
60000/60000 [==============================] - 1s 19us/sample - loss: 0.0748 - accuracy: 0.9765 - val_loss: 0.0900 - val_accuracy: 0.9735
Epoch 6/20
60000/60000 [==============================] - 1s 23us/sample - loss: 0.0624 - accuracy: 0.9804 - val_loss: 0.0964 - val_accuracy: 0.9718
Epoch 7/20
60000/60000 [==============================] [...]<jupyter_text>Performing inference<jupyter_code>valid_0 = X_valid[0].reshape(1, 784)
model.predict(valid_0)
model.predict_classes(valid_0)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16670574443141853, 'fraction_numerical': 0.07784290738569753, 'mean_word_length': 5.493150684931507, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '343480', 'license': 'MIT', 'n_tokens_mistral': 1554, 'n_tokens_neox': 1307, 'n_words': 245, 'path': 'notebooks/deep_net_in_tensorflow.ipynb', 'repo_name': 'data-catalysis/DLTFpT', 'size': '4265'}