<jupyter_start><jupyter_text>CIFAR-10: Object Recognition in Images Using Deep Convolutional Neural Network (DCNN)In this project, we used deep convolutional neural network model trained on CIFAR-10 (Canadian Institute For Advanced Research) dataset to recognize multiple objects present in various images.  1. Import libraries<jupyter_code>import matplotlib.pyplot as plt
import keras
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten, Conv2D, MaxPooling2D 
from keras.layers.normalization import BatchNormalization
from keras.optimizers import SGD
from keras.utils.np_utils import to_categorical<jupyter_output><empty_output><jupyter_text>2. Import CIFAR-10 datasetThe CIFAR-10 dataset (Canadian Institute For Advanced Research) is a collection of images that are commonly used to train machine learning and computer vision algorithms. It is one of the most widely used datasets for machine learning research. The CIFAR-10 dataset contains 60,000 32x32 color images in 10 different classes. The 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. There are 6,000 images of each class.<jupyter_code># import cifar-10 dataset
from keras.datasets import cifar10<jupyter_output><empty_output><jupyter_text>2.1. Load CIFAR-10 dataset<jupyter_code># load cifar-10 dataset
(X_train, y_train), (X_test, y_test) = cifar10.load_data()<jupyter_output><empty_output><jupyter_text>2.2. Examining the dataset<jupyter_code># shape of training data
print("Shape of training data: 'features'", X_train.shape)
print("Shape of training data: 'labels'", y_train.shape)
# shape of test data
print("Shape of test data: 'features'", X_test.shape)
print("Shape of test data: 'labels'", y_test.shape)<jupyter_output>Shape of training data: 'features' (50000, 32, 32, 3)
Shape of training data: 'labels' (50000, 1)
Shape of test data: 'features' (10000, 32, 32, 3)
Shape of test data: 'labels' (10000, 1)<jupyter_text>2.3. CIFAR-10 dataset classesThe 10 different classes represent airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. There are 6,000 images of each class.<jupyter_code># cifar-10 classes
cifar_classes = ['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']<jupyter_output><empty_output><jupyter_text>For each image there is a corresponding label, which is a class index.<jupyter_code>print('Sample training images and their labels: ' + str([z[0] for z in y_train[0:10]])) 
print('Corresponding classes for the labels: ' + str([cifar_classes[z[0]] for z in y_train[0:10]]))<jupyter_output>Sample training images and their labels: [6, 9, 9, 4, 1, 1, 2, 7, 8, 3]
Corresponding classes for the labels: ['frog', 'truck', 'truck', 'deer', 'automobile', 'automobile', 'bird', 'horse', 'ship', 'cat']<jupyter_text>Here are the first 10 sample images from the training data. The images have a structure of (32,32,3) which correspond to (width, height, RGB).<jupyter_code>import matplotlib.pyplot as plt
for z in range(10):
    images = X_train[z]
    plt.imshow(images)
    plt.show()<jupyter_output><empty_output><jupyter_text>2.4. Preparing the datasetWe are going to use the deep convolutional neural network to classify the images. Instead of class indices we will use one-hot encoded vectors to represent the labels of the samples.<jupyter_code>cifar_num_classes = 10
# Transform label indices to one-hot encoded vectors
y_train = to_categorical(y_train, cifar_num_classes)
y_test = to_categorical(y_test, cifar_num_classes)

X_train = X_train.astype('float32')
X_test = X_test.astype('float32')<jupyter_output><empty_output><jupyter_text>2.5. Normalize the image dataA simple way to normalize our data is to scale the pixel values between 0 to 1 range.<jupyter_code># Normalization of pixel values between 0 to 1 range 
X_train /= 255.0
X_test /= 255.0<jupyter_output><empty_output><jupyter_text>3. Training of Convolutional Neural Network (CNN) ModelLet's train our model now! We will store the training loss values and metrics in a history object, so we can visualize the training process later.We are going to train the model for 15 epochs, using a batch size of 64 and a validation split of 0.2. The latter means that 20% of our training data will be used as validation samples. 3.1. Regularization Regularization methods are used to prevent overfitting, making these larger models generalize better.We used "dropout" as a regularization method. Dropout works on a neural network layer by masking a random subset of its outputs (zeroing them) for every input with probability p and scaling up the rest of the outputs by 1/(1 - p).Dropout is normally used during training. Masking prevents gradient backpropagation through the masked outputs. The method thus selects a random subset of the neural network to train on any particular example. This can be thought of as training a model ensemble to solve the task, with the individual models sharing parameters.At test time, p is set to zero. This can be interpreted as averaging the outputs of the ensemble models. Because of the scaling, the expected layer outputs are the same during training and testing. 3.2. Batch NormalizationBatch Normalization works by normalizing layer outputs to a running mean and variance. This speeds up training and improves the final performance of the model. The running statistics are fixed at test time.While batch normalization works as a regularizer, it also benefits smaller models. 3.3 Creating CNN modelWe will use three convolutional layers, each with a kernel size of (3,3) and ReLU activation function. First convolutional layer with 32 filters, second convolutional layer with 64 filters and third convolutional layer with 128 filters.<jupyter_code>model = Sequential()
model.add(Conv2D(32, (3, 3), padding='same', input_shape=(32, 32, 3)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(Conv2D(64, (3, 3), padding='same', input_shape=(32, 32, 3)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.3))

model.add(Conv2D(128, (3, 3), padding='same', input_shape=(32, 32, 3)))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.4))

model.add(Flatten())
model.add(Dense(80))
model.add(BatchNormalization())
model.add(Activation('relu'))
model.add(Dropout(0.3))
model.add(Dense(cifar_num_classes))
model.add(BatchNormalization())
model.add(Activation('softmax'))<jupyter_output><empty_output><jupyter_text>The summary of this model could be seen below:<jupyter_code>model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_10 (Conv2D)           (None, 32, 32, 32)        896       
_________________________________________________________________
batch_normalization_16 (Batc (None, 32, 32, 32)        128       
_________________________________________________________________
activation_16 (Activation)   (None, 32, 32, 32)        0         
_________________________________________________________________
max_pooling2d_10 (MaxPooling (None, 16, 16, 32)        0         
_________________________________________________________________
dropout_13 (Dropout)         (None, 16, 16, 32)        0         
_________________________________________________________________
conv2d_11 (Conv2D)           (None, 16, 16, 64)        18496     
_________________________________________________________________
batch_norm[...]<jupyter_text>We used the Stochastic Gradient Descent algorithm (SGD) to optimize the weights on the backpropagation. Set the momentum parameter as 0.9, and just leave the others as default.<jupyter_code>opt = SGD(lr=0.01, momentum=0.9, decay=0)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
history = model.fit(X_train, y_train, batch_size=64, epochs=100, verbose=2, validation_split=0.2)
def plotAccuracy(history):  
    plt.plot(history.history['acc'])
    plt.plot(history.history['val_acc'])
    plt.title('model Accuracy')
    plt.ylabel('acc')
    plt.xlabel('epoch')
    plt.legend(['train', 'validation'], loc='upper left')
    plt.show()
plotAccuracy(history)
def plotLosses(history):  
    plt.plot(history.history['loss'])
    plt.plot(history.history['val_loss'])
    plt.title('model loss')
    plt.ylabel('loss')
    plt.xlabel('epoch')
    plt.legend(['train', 'validation'], loc='upper left')
    plt.show()
plotLosses(history)<jupyter_output><empty_output><jupyter_text>3.4 Evaluating the CNN model<jupyter_code>evaluate = model.evaluate(X_test, y_test, batch_size=128, verbose=0)
print(model.metrics_names)
print(evaluate)<jupyter_output>['loss', 'acc']
[0.5546971242904664, 0.8089]
{'fraction_non_alphanumeric': 0.10514070463106581, 'fraction_numerical': 0.03600709062707733, 'mean_word_length': 5.777027027027027, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '643991', 'license': 'MIT', 'n_tokens_mistral': 2895, 'n_tokens_neox': 2632, 'n_words': 922, 'path': 'CIFAR-10: Object Recognition in Images Using Deep Convolutional Neural Network (DCNN).ipynb', 'repo_name': 'ZainAmin/CIFAR_10_Object_Recognition_in_Images_Using_Deep_Convolutional_Neural_Network_-DCNN-', 'size': '9026'}