<jupyter_start><jupyter_text>Building a CNN to classify images on the CIFAR dataset https://www.cs.toronto.edu/~kriz/cifar.html|The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class.  Building a CNNWe will check how different layers are configured then build our own model<jupyter_code># To ignore warnings
import warnings
warnings.filterwarnings("ignore")
# Deep learning framework
import keras
from keras.datasets import cifar10
from keras.preprocessing.image import ImageDataGenerator
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D
# For plotting
import matplotlib.pyplot as plt
%matplotlib inline

# import sys
# sys.path.append("..")
# from Scripts.utils import plot_loss_accuracy
# Loading the data and splitting it to train and test
(x_train, y_train), (x_test, y_test) = cifar10.load_data()
print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')
# Lets look at one of the images and its label
print(y_train[0])
plt.imshow(x_train[0])
classes = 10

y_train = keras.utils.np_utils.to_categorical(y_train, classes)
y_test = keras.utils.np_utils.to_categorical(y_test, classes)
print(y_train[0])
# Making everything to be float and scale
x_train = x_train.astype('float32')
x_test = x_test.astype('float32')
x_train /= 255
x_test /= 255<jupyter_output><empty_output><jupyter_text>Conv2D```pythonkeras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)```When using this layer as the first layer in a model, provide the keyword argument input_shape- `filters`: Integer, the number of output filters in the convolution.- `kernel_size`: An integer or tuple/list of 2 integers, specifying the height and width of the 2D convolution window.- `strides`: and (x,y) tuple giving the stride in each dimension.  Default is `(1,1)`- `activation`: Activation function to use. If you don't specify anything, no activation is applied.Note, the size of the output will be determined by the kernel_size, strides MaxPooling2D```pythonkeras.layers.MaxPooling2D(pool_size=(2, 2), strides=None, padding='valid', data_format=None)```- `pool_size`: Integer or tuple of 2 integers, factors by which to downscale (vertical, horizontal).- `strides`: Integer, tuple of 2 integers, or None. Strides values. Flatten```pythonkeras.layers.Flatten(data_format=None)```Turns its input into a one-dimensional vector (per instance). Usually used when transitioning between convolutional layers and fully connected layers. Activation```pythonkeras.layers.Activation(activation)```Applies an activation function to the output SequentialThe Sequential model is a linear stack of layers. You can create a Sequential model by passing a list of layer instances to the constructor<jupyter_code>model_1 = Sequential()


model_1.add(Conv2D(64, (4, 4), strides = (2,2), padding='same', input_shape=x_train.shape[1:]))
model_1.add(Activation('relu'))
model_1.add(Conv2D(64, (4, 4), strides = (2,2)))
model_1.add(Activation('relu'))
model_1.add(MaxPooling2D(pool_size=(2, 2)))
model_1.add(Dropout(0.3))
model_1.add(Flatten())
model_1.add(Dense(512))
model_1.add(Activation('relu'))
model_1.add(Dropout(0.5))
model_1.add(Dense(classes))
model_1.add(Activation('softmax'))

model_1.summary()
opt = keras.optimizers.Adam(lr=0.0005, decay=1e-6)
model_1.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
history = model_1.fit(x_train, y_train, batch_size=128, epochs=10, validation_data=(x_test, y_test), shuffle=True)
score = model_1.evaluate(x_test, y_test, verbose=0)
print('Test loss:', score[0])
print('Test accuracy:', score[1])
def plot_loss_accuracy(history):
    fig = plt.figure(figsize=(12, 6))
    ax = fig.add_subplot(1, 2, 1)
    ax.plot(history.history["loss"],'r-d', label="Train Loss")
    ax.plot(history.history["val_loss"],'b-d', label="Validation Loss")
    ax.legend()
    ax.set_title('cross_entropy loss')
    ax.grid(True)

    ax = fig.add_subplot(1, 2, 2)
    ax.plot(history.history["accuracy"],'r-d', label="Train Accuracy")
    ax.plot(history.history["val_accuracy"],'b-d', label="Validation Accuracy")
    ax.legend()
    ax.set_title('accuracy')
    ax.grid(True)
plot_loss_accuracy(history)<jupyter_output><empty_output><jupyter_text>ExerciseOur previous model had the structure:Conv -> Conv -> MaxPool -> Flatten -> Dense -> Final Classification(with appropriate activation functions and dropouts)1. Build a more complicated model with the following pattern:- Conv -> Conv -> Conv -> MaxPool -> Conv -> Conv -> MaxPool -> Flatten -> Dense -> Dense -> Final Classification- Use strides of 1 for all convolutional layers.2. How many parameters does your model have<jupyter_code>datagen = ImageDataGenerator(
    featurewise_center=False,  # set input mean to 0 over the dataset
    samplewise_center=False,  # set each sample mean to 0
    featurewise_std_normalization=False,  # divide inputs by std of the dataset
    samplewise_std_normalization=False,  # divide each input by its std
    zca_whitening=False,  # apply ZCA whitening
    rotation_range=0,  # randomly rotate images in the range (degrees, 0 to 180)
    width_shift_range=0.1,  # randomly shift images horizontally (fraction of total width)
    height_shift_range=0.1,  # randomly shift images vertically (fraction of total height)
    horizontal_flip=True,  # randomly flip images
    vertical_flip=True)  # randomly flip images
                   
history = datagen.fit(x_train)      
# This computes any statistics that may be needed (e.g. for centering) from the training set.

    
# Fit the model on the batches generated by datagen.flow().
model_1.fit_generator(datagen.flow(x_train, y_train,batch_size=128), steps_per_epoch=x_train.shape[0] // 128, epochs=10, validation_data=(x_test, y_test))
print(history)
plot_loss_accuracy(model_1.history)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1137091607943626, 'fraction_numerical': 0.023542600896860985, 'mean_word_length': 5.840087623220153, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 28, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '306873', 'license': 'MIT', 'n_tokens_mistral': 2109, 'n_tokens_neox': 1965, 'n_words': 573, 'path': '3rdMonth{deep_learning}/Notebook/02-cifar-cnn-to-classify.ipynb', 'repo_name': 'WilliamOtieno/computer-vision-ai-saturdays', 'size': '6244'}