<jupyter_start><jupyter_text>Dataset loadingIn the first step we prepare everything and load our dataset with handwritten digits. Our goal is to load the image into our program and classify it. Classifying means to recognize which digit it is. Is it a *0* or a *9*?A small hint  signals a comment in the code, so programmers note hints to understand lines of code easier ;-)<jupyter_code># TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras
import numpy as np
import matplotlib.pyplot as plt
print(tf.__version__)

# We are loading the data
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()

# We normalize the images, thus that it contains values between [0 - 1]. This is prefereable for NNs.
train_images = np.expand_dims(train_images / 255.0, -1)
test_images = np.expand_dims(test_images / 255.0, -1)<jupyter_output><empty_output><jupyter_text>Visualize - Illustrate - PictorializeIn the next step, we load a *0* and a *9* from our training dataset and visualize the two digits.<jupyter_code># Load a 0 from the training data
indicies_of_all_0 = (np.where(test_labels == 0))[0]
image_with_0 = test_images[indicies_of_all_0[0]]

# Lade eine 9 aus den Trainingsdaten
indicies_of_all_9 = (np.where(test_labels == 9))[0]
image_with_9 = test_images[indicies_of_all_9[0]]

# Visualisieren (= anzeigen) der Bilder, damit wir auch sehen ob wir das ganze richtig geladen haben 
plt.figure()
plt.imshow(image_with_0[:,:,0], cmap=plt.cm.binary)
plt.title("This is a 0")
plt.show()

plt.figure()
plt.imshow(image_with_9[:,:,0], cmap=plt.cm.binary)
plt.title("This is a 9")
plt.show()<jupyter_output><empty_output><jupyter_text>Define neural networkNext we need to define the architecture of our neural network. How many layers should it have, how many neurons do these layers have.We first decide on the following architecture:*   Input Layer: 28x28 (this is the size of our input images!)*   3 x Convolutionial layer with pooling layer *   Fully Connected Network (FCN) layer (called *dense* in TF!) with 128 neurons with ReLU activation*   Output are 10 neurons (we have 10 digits we want to classify)<jupyter_code># network architecture
model = keras.Sequential([
    keras.layers.Conv2D(8, (3, 3), activation='relu', input_shape=(28,28,1)),
    keras.layers.MaxPooling2D((2, 2)),
    keras.layers.Conv2D(16, (3, 3), activation='relu'),
    keras.layers.MaxPooling2D((2, 2)),
    keras.layers.Conv2D(16, (3, 3), activation='relu'),
    keras.layers.Flatten(),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dense(10)
])
# Let TF build the network
model.compile(optimizer='adam',
              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
              metrics=['accuracy'])<jupyter_output><empty_output><jupyter_text>Train the neural networkIn the next step we train our network with the data we loaded above. Training is also called *fitting*, because during training the weights of the neurons are adjusted, i.e. they are fitted. The word comes from English! Of course we have to tell TF how long the network should be trained. This is expressed by how often the training data should be shown to the network. * 1 x show all training data = 1 epoch* 2 x show all training data = 2 epochs<jupyter_code># Train the network for 5 epochs
model.fit(train_images, train_labels, epochs=5)

#Save my model
# 1) select top level drive Folder in Browser & create directory with name modul_2_cnn --> 'right click' -> new Folder -> modul_2_cnn'
# from google.colab import drive
# drive.mount('/content/drive')
# tf.saved_model.save(model, '/content/drive/My Drive/modul_2_cnn/model')<jupyter_output><empty_output><jupyter_text>Check how good the network isWe have trained the network, now we also want to know how well it works. We also say we *evaluate* the network now. We evaluate with the test data. We ask how many of the test data are correctly classified, that is, how often the network correctly recognizes the number.<jupyter_code># Testing the network
test_loss, test_acc = model.evaluate(test_images,  test_labels, verbose=0)
print('Our result:')
print('Out of ', test_images.shape[0], ' we correctly classified ', int(test_acc * test_images.shape[0]), '. These are {:.2f}% of the data'.format(test_acc * 100.0))<jupyter_output><empty_output><jupyter_text>Can we visualize the NN layers?What happens inside the *convolutional layers*? How do the filters look like? That's what we visualize in the following code blocks<jupyter_code>conv1_layer_weight = model.layers[0].get_weights()[0][:,:,0,:]

import matplotlib.gridspec as gridspec
# First convolutional layer --> 8 Filter
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(2,4, wspace=0.05, hspace=0.05)
for idx in range(8):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(conv1_layer_weight[:,:,idx], cmap=plt.cm.binary)
conv2_layer_weight = model.layers[2].get_weights()[0][:,:,0,:]

import matplotlib.gridspec as gridspec
# Second convolutional layer --> 16 Filter
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(4,4, wspace=0.05, hspace=0.05)
for idx in range(16):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(conv2_layer_weight[:,:,idx], cmap=plt.cm.binary)
conv3_layer_weight = model.layers[4].get_weights()[0][:,:,0,:]

import matplotlib.gridspec as gridspec
# Third convolution layer --> 16 Filter
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(4,4, wspace=0.05, hspace=0.05)
for idx in range(16):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(conv3_layer_weight[:,:,idx], cmap=plt.cm.binary)<jupyter_output><empty_output><jupyter_text>Can we visualize the effects on the image?We visualize the effects of the filters in the three layer on the 0 image.<jupyter_code>print(model.summary())

layer_outputs = [layer.output for layer in model.layers[:6]] 
cnn_model = tf.keras.Model(inputs=model.inputs, outputs=layer_outputs)
activation_of_zero = cnn_model.predict(np.expand_dims(image_with_0, axis=0))
print(len(activation_of_zero))


import matplotlib.gridspec as gridspec
# First convolutional layer --> 8 Filter
output_conv_1 = activation_of_zero[0]
print('Activaiton of first layer with shape: ', output_conv_1.shape)
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(2,4, wspace=0.05, hspace=0.05)
for idx in range(8):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(output_conv_1[0,:,:,idx], cmap=plt.cm.binary)

# Second convolutional layer --> 16 Filter
output_conv_2 = activation_of_zero[2]
print('Activaiton of second layer with shape: ', output_conv_2.shape)
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(4,4, wspace=0.05, hspace=0.05)
for idx in range(16):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(output_conv_2[0,:,:,idx], cmap=plt.cm.binary)
# Dritter convolution layer --> 16 Filter
output_conv_3 = activation_of_zero[4]
print('Activaiton of third layer with shape: ', output_conv_3.shape)
fig = plt.figure(figsize=(10,10))
gs1 = gridspec.GridSpec(4,4, wspace=0.05, hspace=0.05)
for idx in range(16):
    a = plt.subplot(gs1[idx])
    a.axis('off')
    imgplot = plt.imshow(output_conv_3[0,:,:,idx], cmap=plt.cm.binary)<jupyter_output><empty_output><jupyter_text>Can you find out the following.* Training time (= epochs) of the neural network:  * What happens if you train only for a very short time (e.g.: 1 epoch)? How many of the test data are then still recognized correctly?  * What happens if you train for a long time (e.g. 1000 epochs)? How many of the test data will be recognized correctly then? What can you observe?  * **Tip**: Find the place in the code where you train and change the number of epochs accordingly. * What happens if you shift the input number slightly to the left? Will it still be recognized correctly? Just try the example and describe what you see. Can you find an explanation for it?* What happens if the input number is slightly noisy? Is it still recognized correctly? Just try the example and describe what you see. Can you find an explanation for it? Where could noise come from, for example, can you find examples of it?<jupyter_code># Example of shifted 9

shifted_nine = np.zeros_like(image_with_9) # we create an empty image of the same size as the 9
shifted_nine[:, :15] = image_with_9[:, 8:23]

plt.figure()
plt.imshow(image_with_9[:,:,0], cmap=plt.cm.binary)
plt.title("This is the correct 9")
plt.show()

plt.figure()
plt.imshow(shifted_nine[:,:,0], cmap=plt.cm.binary)
plt.title("This is the shifted 9")
plt.show()

from scipy.special import softmax
logits_of_nine = model.predict(np.expand_dims(image_with_9, 0))
probabilities_of_nine = softmax(logits_of_nine)[0]
detected_class_of_nine = np.argmax(probabilities_of_nine)
print('The NN classified the 9 as ', detected_class_of_nine, ' with a probability of ', probabilities_of_nine[detected_class_of_nine])

logits_of_shifted_nine = model.predict(np.expand_dims(shifted_nine, 0))
probabilities_of_shifted_nine = softmax(logits_of_shifted_nine)[0]
detected_class_of_shifted_nine = np.argmax(probabilities_of_shifted_nine)
print('The NN classified the shifted 9 as ', detected_class_of_shifted_nine, ' with a probability of ', probabilities_of_shifted_nine[detected_class_of_shifted_nine])

# Example of noised 9

noised_nine = np.copy(image_with_9) 
noise = np.zeros_like(image_with_9) 
image_coordinates = [np.random.randint(0, i - 1, 50) for i in noise[:,:,0].shape]
noise[image_coordinates] = 1
noised_nine += noise
image_coordinates = [np.random.randint(0, i - 1, 50) for i in noise[:,:,0].shape]
noise[image_coordinates] = -1
noised_nine += noise
noised_nine = np.clip(noised_nine,0,1)


plt.figure()
plt.imshow(image_with_9[:,:,0], cmap=plt.cm.binary)
plt.title("This is the correct 9")
plt.show()

plt.figure()
plt.imshow(noised_nine[:,:,0], cmap=plt.cm.binary)
plt.title("This is the noised 9")
plt.show()

from scipy.special import softmax
logits_of_nine = model.predict(np.expand_dims(image_with_9, 0))
probabilities_of_nine = softmax(logits_of_nine)[0]
detected_class_of_nine = np.argmax(probabilities_of_nine)
print('The NN classified the 9 as ', detected_class_of_nine, ' with a probability of ', probabilities_of_nine[detected_class_of_nine])

logits_of_noised_nine = model.predict(np.expand_dims(noised_nine, 0))
probabilities_of_noised_nine = softmax(logits_of_noised_nine)[0]
detected_class_of_noised_nine = np.argmax(probabilities_of_noised_nine)
print('The NN classified the noised 9 as ', detected_class_of_noised_nine, ' with a probability of ', probabilities_of_noised_nine[detected_class_of_noised_nine])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11394763812540762, 'fraction_numerical': 0.026926302059070158, 'mean_word_length': 5.605538461538462, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 42, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '320532', 'license': 'MIT', 'n_tokens_mistral': 3887, 'n_tokens_neox': 3656, 'n_words': 1076, 'path': 'english_version/modul_2_mnist-digits_cnn.ipynb', 'repo_name': 'hacker-school/juml', 'size': '10733'}