<jupyter_start><jupyter_text>This is based on the quickstart for experts tutiorial from the official tensorFlow 2 tutiorials. We start by importing TensorFlow and Keras into the programm.<jupyter_code>import tensorflow as tf

from tensorflow.keras.layers import Dense, Flatten, Conv2D
from tensorflow.keras import Model<jupyter_output><empty_output><jupyter_text>Now we load the MNIST dataset and prepare it.<jupyter_code>mnist = tf.keras.datasets.mnist

(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0

# Add a channels dimension
x_train = x_train[..., tf.newaxis]
x_test = x_test[..., tf.newaxis]<jupyter_output><empty_output><jupyter_text>Shuffle the data.<jupyter_code>train_ds = tf.data.Dataset.from_tensor_slices(
    (x_train, y_train)).shuffle(10000).batch(32)

test_ds = tf.data.Dataset.from_tensor_slices((x_test, y_test)).batch(32)<jupyter_output><empty_output><jupyter_text>Now we build a model using Keras.<jupyter_code>class MyModel(Model):
    def __init__(self):
        super(MyModel, self).__init__()
        self.conv1 = Conv2D(32, 3, activation = 'relu')
        self.flatten = Flatten()
        self.d1 = Dense(128, activation = 'relu')
        self.d2 = Dense(10)
    
    def call(self, x):
        x = self.conv1(x)
        x = self.flatten(x)
        x = self.d1(x)
        return self.d2(x)
    
# Create instance of the model
model = MyModel()<jupyter_output><empty_output><jupyter_text>We choose now an optimzer and a loss function.<jupyter_code>loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits = True)

optimizer = tf.keras.optimizers.Adam()<jupyter_output><empty_output><jupyter_text>To measure the models loss and the accuracy metrics need to be selected. The idea is that over these metrics values are changed towards an optimal state in a number of iterations (epochs).<jupyter_code>train_loss = tf.keras.metrics.Mean(name='train_loss')
train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='train_accuracy')

test_loss = tf.keras.metrics.Mean(name='test_loss')
test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(name='test_accuracy')<jupyter_output><empty_output><jupyter_text>To train the model we utilise tf.GradientTape.<jupyter_code>@tf.function
def train_step(images, labels):
    with tf.GradientTape() as tape:
        # training=True is only needed if there are layers with different
        # behavior during training versus inference (e.g. Dropout).
        predictions = model(images, training=True)
        loss = loss_object(labels, predictions)
    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))

    train_loss(loss)
    train_accuracy(labels, predictions)<jupyter_output><empty_output><jupyter_text>Test the model.<jupyter_code>@tf.function
def test_step(images, labels):
    # training = False is only needed if there are layers with different
    # behavior predictions = model(images, training=False)
    predictions = model(images, training = False)
    t_loss = loss_object(labels, predictions)
    
    test_loss(t_loss)
    test_accuracy(labels, predictions)
EPOCHS = 5

for epoch in range(EPOCHS):
    # Reset the metrics at the start of the next epoch
    train_loss.reset_states()
    train_accuracy.reset_states()
    test_loss.reset_states()
    test_accuracy.reset_states()
    
    for images, labels in train_ds:
        train_step(images, labels)
    
    for test_images, test_labels in test_ds:
        test_step(test_images, test_labels)
    
    template = 'Epoch: {}, Loss: {}, Accuracy: {}, Test Loss: {}, Test Accuracy: {}'
    print(template.format(
        epoch + 1,
        train_loss.result(),
        train_accuracy.result() * 100,
        test_loss.result(),
        test_accuracy.result() * 100
    ))<jupyter_output>Epoch: 1, Loss: 0.04364383965730667, Accuracy: 98.65666961669922, Test Loss: 0.04782670736312866, Test Accuracy: 98.37999725341797
Epoch: 2, Loss: 0.02284659631550312, Accuracy: 99.26666259765625, Test Loss: 0.051318977028131485, Test Accuracy: 98.38999938964844
Epoch: 3, Loss: 0.014051279053092003, Accuracy: 99.51666259765625, Test Loss: 0.06033865734934807, Test Accuracy: 98.22999572753906
Epoch: 4, Loss: 0.009403767064213753, Accuracy: 99.68499755859375, Test Loss: 0.06457576900720596, Test Accuracy: 98.29000091552734
Epoch: 5, Loss: 0.006028480362147093, Accuracy: 99.80999755859375, Test Loss: 0.06443192809820175, Test Accuracy: 98.23999786376953
{'fraction_non_alphanumeric': 0.10303830911492734, 'fraction_numerical': 0.08608542492294144, 'mean_word_length': 4.650497512437811, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '131915', 'license': 'MIT', 'n_tokens_mistral': 1830, 'n_tokens_neox': 1563, 'n_words': 354, 'path': 'Notebooks/mnist_advanced.ipynb', 'repo_name': 'Oseanus/tf2-playground', 'size': '4542'}