<jupyter_start><jupyter_text>Ungraded Lab: Improving Computer Vision Accuracy using Convolutions  Shallow Neural Network In the previous lessons, you saw how to do fashion recognition using a neural network containing three layers -- the input layer (in the shape of the data), the output layer (in the shape of the desired output) and only one hidden layer. You experimented with the impact of different sizes of hidden layer, number of training epochs etc on the final accuracy. For convenience, here's the entire code again. Run it and take a note of the test accuracy that is printed out at the end.<jupyter_code># Import all package I will need
%matplotlib inline
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import datasets
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns 

plt.style.use('ggplot')
# create dataset
(training_images, training_labels), (test_images, test_labels) = datasets.fashion_mnist.load_data()

print(f'shape of training_images: {training_images.shape}')
print(f'shape of training_labels: {training_labels.shape}')
print(f'shape of test_images: {test_images.shape}')
print(f'shape of test_labels: {test_labels.shape}')

# Create first models without conv2d
model_one = keras.Sequential([
    keras.layers.Flatten(input_shape = (training_images.shape[1], training_images.shape[2])),
    keras.layers.Dense(128, activation = tf.nn.relu),
    keras.layers.Dense(10, activation = tf.nn.softmax)
])
model_one.compile(
    optimizer = 'adam',
    loss = 'sparse_categorical_crossentropy',
    metrics = ['accuracy']
)
# train model_one
model_one.fit(training_images, training_labels, epochs=5)
# Evaluate model
model_one.evaluate(test_images, test_labels)<jupyter_output>313/313 [==============================] - 1s 1ms/step - loss: 0.5891 - accuracy: 0.7928<jupyter_text>Convolutional Neural Network In the model above, your accuracy will probably be about 89% on training and 87% on validation. Not bad. But how do you make that even better? One way is to use something called convolutions. We're not going into the details of convolutions in this notebook (please see resources in the classroom), but the ultimate concept is that they narrow down the content of the image to focus on specific parts and this will likely improve the model accuracy.If you've ever done image processing using a filter (like [this](https://en.wikipedia.org/wiki/Kernel_(image_processing))), then convolutions will look very familiar. In short, you take an array (usually 3x3 or 5x5) and scan it over the entire image. By changing the underlying pixels based on the formula within that matrix, you can do things like edge detection. So, for example, if you look at the above link, you'll see a 3x3 matrix that is defined for edge detection where the middle cell is 8, and all of its neighbors are -1. In this case, for each pixel, you would multiply its value by 8, then subtract the value of each neighbor. Do this for every pixel, and you'll end up with a new image that has the edges enhanced.This is perfect for computer vision because it often highlights features that distinguish one item from another. Moreover, the amount of information needed is then much less because you'll just train on the highlighted features.That's the concept of **Convolutional Neural Networks**. Add some layers to do convolution before you have the dense layers, and then the information going to the dense layers is more focused and possibly more accurate.Run the code below. This is the same neural network as earlier, but this time with [Convolution](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D) and [MaxPooling](https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D) layers added first. It will take longer, but look at the impact on the accuracy.<jupyter_code># create model with Conv2d
model_two = keras.Sequential([
    # Conv2D 
    keras.layers.Conv2D(
        64, 
        (3,3), 
        activation = tf.nn.relu, 
        input_shape = (training_images.shape[1], training_images.shape[2], 1)
    ),
    keras.layers.MaxPooling2D(2, 2),
    keras.layers.Conv2D(
        64,
        (3,3),
        activation = tf.nn.relu
    ),
    keras.layers.MaxPooling2D(2,2),
    
    keras.layers.Flatten(input_shape = (training_images[1], training_images.shape[2])),
    keras.layers.Dense(128, activation = tf.nn.relu),
    keras.layers.Dense(10, activation = tf.nn.softmax)
])
model_two.compile(
    optimizer = 'adam',
    loss = 'sparse_categorical_crossentropy',
    metrics = ['accuracy']
)
model_one.summary()
# training the model_two
model_two.fit(training_images, training_labels, epochs=5)
## evaluate model_two
model_two.evaluate(test_images, test_labels)<jupyter_output>313/313 [==============================] - 3s 10ms/step - loss: 0.3369 - accuracy: 0.8808<jupyter_text>It's likely gone up to about 92% on the training data and 90% on the validation data. That's significant, and a step in the right direction!Look at the code again, and see, step by step how the convolutions were built. Instead of the input layer at the top, you added a [Conv2D layer](https://www.tensorflow.org/api_docs/python/tf/keras/layers/Conv2D). The parameters are: 1. The number of convolutions you want to generate. The value here is purely arbitrary but it's good to use powers of 2 starting from 32. 2. The size of the Convolution. In this case, a 3x3 grid. 3. The activation function to use. In this case, you used a ReLU, which you might recall is the equivalent of returning x when x>0, else return 0. 4. In the first layer, the shape of the input data.  You'll follow the convolution with a [MaxPool2D](https://www.tensorflow.org/api_docs/python/tf/keras/layers/MaxPool2D) layer which is designed to compress the image, while maintaining the content of the features that were highlighted by the convlution. By specifying (2,2) for the MaxPooling, the effect is to quarter the size of the image. Without going into too much detail here, the idea is that it creates a 2x2 array of pixels, and picks the biggest one. Thus, it turns 4 pixels into 1. It repeats this across the image, and in doing so, it halves both the number of horizontal and vertical pixels, effectively reducing the image to 25% of the original image. You can call model.summary() to see the size and shape of the network, and you'll notice that after every max pooling layer, the image size is reduced in this way.model = tf.keras.models.Sequential([              tf.keras.layers.Conv2D(32, (3,3),               activation='relu', input_shape=(28, 28, 1)),              tf.keras.layers.MaxPooling2D(2, 2),              After this, you'll just have the same DNN structure as the non convolutional version. The same 128 dense layers, and 10 output layers as in the pre-convolution example:              tf.keras.layers.Dense(128, activation='relu'),              tf.keras.layers.Dense(10, activation='softmax')            ])  About overfitting Try running the training for more epochs -- say about 20, and explore the results. But while the results might seem really good, the validation results may actually go down, due to something called overfitting. In a nutshell, overfitting occurs when the network learns the data from the training set really well, but it's too specialised to only that data, and as a result is less effective at interpreting other unseen data. For example, if all your life you only saw red shoes, then when you see a red shoe you would be very good at identifying it. But blue suede shoes might confuse you... and you know you should never mess with my blue suede shoes.  Visualizing the Convolutions and Pooling Let's explore how to show the convolutions graphically. The cell below prints the first 100 labels in the test set, and you can see that the ones at index 0, index 23 and index 28 are all the same value (i.e. 9). They're all shoes. Let's take a look at the result of running the convolution on each, and you'll begin to see common features between them emerge. Now, when the dense layer is training on that data, it's working with a lot less, and it's perhaps finding a commonality between shoes based on this convolution/pooling combination.<jupyter_code>print(test_labels[:100])
f, axarr = plt.subplots(3,4)

FIRST_IMAGE=0
SECOND_IMAGE=23
THIRD_IMAGE=28
CONVOLUTION_NUMBER = 1

layer_outputs = [layer.output for layer in model_two.layers]
activation_model = tf.keras.models.Model(inputs = model_two.input, outputs = layer_outputs)

for x in range(0,4):
    f1 = activation_model.predict(test_images[FIRST_IMAGE].reshape(1, 28, 28, 1))[x]
    axarr[0,x].imshow(f1[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
    axarr[0,x].grid(False)
  
    f2 = activation_model.predict(test_images[SECOND_IMAGE].reshape(1, 28, 28, 1))[x]
    axarr[1,x].imshow(f2[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
    axarr[1,x].grid(False)
  
    f3 = activation_model.predict(test_images[THIRD_IMAGE].reshape(1, 28, 28, 1))[x]
    axarr[2,x].imshow(f3[0, : , :, CONVOLUTION_NUMBER], cmap='inferno')
    axarr[2,x].grid(False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08673748767933413, 'fraction_numerical': 0.021574854889935383, 'mean_word_length': 4.400354819633353, 'pattern_counts': {'":': 0, '<': 11, '<?xml version=': 0, '>': 12, 'https://': 5, 'lorem ipsum': 0, 'www.': 4, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '373022', 'license': 'MIT', 'n_tokens_mistral': 2755, 'n_tokens_neox': 2569, 'n_words': 1131, 'path': 'lab/C1_W3_Lab_1_improving_accuracy_using_convolutions.ipynb', 'repo_name': 'camara94/introduction_to_tensorflow_ai', 'size': '9131'}