<jupyter_start><jupyter_text>**Dphi****Deep Learning Bootcamp - Assignment 2 - Intermediate: Recognize Animals****Submitted by :** ***<NAME>***Image recognition is a vital component in robotics such as driverless vehicles or domestic robots. Image recognition is also important in image search engines such as Google or Bing image search whereby you use rich image content to query for similar stuff. Like in Google photos where the system uses image recognition to categorize your images into things like cats, dogs, people and so on so that you can quickly search your albums for things like, “give me photos of my cat”, that's awesome.Ever noticed how Facebook instantly recognises your friend’s face and asks you if you want to tag him in that photo? That’s image recognition. That’s just a basic example.**Objective**You are working on a robotics project where you are required to train your robot so that it can differentiate between 5 animals or birds. Your task here is to build a deep learning model that helps you recognize the animal or bird in images.**About the Data**The training dataset consists of about 9k medium quality animal images belonging to 5 categories: butterfly, sheep, cow, squirrel, elephant. - mucca (cow), pecora (sheep), elefante (elephant), farfalla (butterfly) and scoiattolo (squirrel). All the images have been collected from "google images" and have been checked by human. There is some erroneous data to simulate real conditions (eg. images taken by users of your app).***Dataset Link:***https://drive.google.com/file/d/176E-pLhoxTgWsJ3MeoJQV_GXczIA6g8D/view?usp=sharingFrom the above link you will be able to download a zip file named ‘animal_dataset_intermediate.zip’. After you extract this zip file, you will get three files:**train** - contains five folders each folder containing images around 1000 to 2000 of those 5 five animals.  Each image has a unique name.**test** - contains 910 random images of those 5 animals whose predictions you are to submit on DPhi platform.**Testing_set_animals.csv** - this is the order of the predictions for each image that is to be submitted on the platform. Make sure the predictions you download are with their image’s filename in the same order as given in this file. **Evaluation Criteria**Submissions are evaluated using Accuracy Score.<jupyter_code># TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout,Flatten
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import SparseCategoricalCrossentropy
from tensorflow.keras.callbacks import TensorBoard,EarlyStopping
from keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.callbacks import ReduceLROnPlateau
from tensorflow.keras.optimizers import SGD,Adam
%load_ext tensorboard

# Helper libraries
import numpy as np
import random
import matplotlib.pyplot as plt
import datetime

print(tf.__version__)
import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.experimental.list_physical_devices('GPU')))<jupyter_output>Num GPUs Available:  1<jupyter_text>**Importing Datasets**<jupyter_code>from google_drive_downloader import GoogleDriveDownloader as gdd

gdd.download_file_from_google_drive(file_id='176E-pLhoxTgWsJ3MeoJQV_GXczIA6g8D', 
                                    dest_path='./data/animal_dataset_intermediate.zip', unzip=True)
import os
DATA_PATH = './data/animal_dataset_intermediate/train/'
print('Cleaning Folder Names')
for i in os.listdir(DATA_PATH):
  os.renames(DATA_PATH+i,DATA_PATH+i.split('_')[0])
#importing necessary libraries
import glob
import cv2
import numpy as np
import matplotlib.pyplot as plt
class_names = ['elefante','farfalla','mucca','pecora','scoiattolo']
IMG_SIZE = 256
import pandas as pd
test_data = pd.read_csv('./data/animal_dataset_intermediate/Testing_set_animals.csv')
test_data<jupyter_output><empty_output><jupyter_text>**Locating the filename in Test Data set with the path**<jupyter_code>test_data['target'] = "./data/animal_dataset_intermediate/test/" + test_data['filename']
test_data

data_test = []

try:
    #files = glob.glob("./data/animal_dataset_intermediate/test"+"/*") # get files in each folder(class)
    files = test_data['target']
    for f in files:
      img = cv2.imread(f) #read the image
      img = cv2.resize(img,(IMG_SIZE,IMG_SIZE)) #resize the image
      data_test.append([np.array(img)]) #Append images and corresponding labels to data
except:
    pass   
    
type(data_test)<jupyter_output><empty_output><jupyter_text>**Converting Test Data set to tensors and reshaping it to 256 * 256 * 3 pixels [Converting into Large pixels improved the learning and gave good accuracy]**<jupyter_code>data_test_tf = tf.convert_to_tensor(data_test, np.float32)
print(type(data_test_tf))
data_test_tf[0]
data_test_tf_reshaped = tf.reshape(data_test_tf,[910,256,256,3])
data_test_tf_reshaped.shape<jupyter_output><empty_output><jupyter_text>**Noramalizing**<jupyter_code>data_test_normalized = np.divide(data_test_tf_reshaped, 255.0)
data_test_normalized[0]<jupyter_output><empty_output><jupyter_text>**Augmentating Train Dataset and have used flow_from_directory for reading according to class based on the folder structure.**<jupyter_code>train_datagen = ImageDataGenerator(rescale = 1./255., # rescaling
                                   rotation_range = 40,  # for augmentation
                                   width_shift_range = 0.2,
                                   validation_split = 0.2,
                                   height_shift_range = 0.2,
                                   shear_range = 0.2,
                                   zoom_range = 0.2,
                                   horizontal_flip = True)

val_datagen = ImageDataGenerator(rescale = 1.0/255.,validation_split = 0.2)

train_generator = train_datagen.flow_from_directory("./data/animal_dataset_intermediate/train",
                                                    batch_size = 32,
                                                    subset="training",
                                                    class_mode = 'categorical', 
                                                    target_size = (256, 256))

val_generator = val_datagen.flow_from_directory("./data/animal_dataset_intermediate/train",
                                                    batch_size = 32,
                                                    subset = "validation",
                                                    class_mode = 'categorical', 
                                                    target_size = (256, 256))

lrr= ReduceLROnPlateau(
                       monitor='val_loss', #Metric to be measured
                       factor=.01, #Factor by which learning rate will be reduced
                       patience=3,  #No. of epochs after which if there is no improvement in the val_loss, the learning rate is reduced
                       min_lr=1e-5) #The minimum learning rate 
learn_rate=.001

sgd=SGD(lr=learn_rate,momentum=.9,nesterov=False)
adam=Adam(lr=learn_rate, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0, amsgrad=False)<jupyter_output><empty_output><jupyter_text>**Function for plotting accuracy and loss**<jupyter_code># plotting helper function
def plotaccuracy(hist):
    plt.plot(hist.history['accuracy'])
    plt.plot(hist.history['val_accuracy'])
    plt.title('model accuracy')
    plt.ylabel('accuracy')
    plt.xlabel('epoch')
    plt.legend(['train', 'validation'], loc='upper left')
    plt.show()
# plotting helper function
def plotloss(hist):
    plt.plot(hist.history['loss'])
    plt.plot(hist.history['val_loss'])
    plt.title('model loss')
    plt.ylabel('loss')
    plt.xlabel('epoch')
    plt.legend(['train', 'validation'], loc='upper left')
    plt.show()<jupyter_output><empty_output><jupyter_text>**Ordinary Convolution Model**<jupyter_code>INPUT_SHAPE = (256, 256, 3)

# define sequential model
model = tf.keras.models.Sequential()

# Add the vgg convolutional base model
#model.add(vgg_layers)

model.add(tf.keras.layers.Conv2D(filters=32, kernel_size=(4, 4), strides=(1, 1), 
                                activation='relu', padding='same', input_shape=INPUT_SHAPE))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))
# define conv-pool layers - set 2
model.add(tf.keras.layers.Conv2D(filters=16, kernel_size=(3, 3), strides=(1, 1), 
                                activation='relu', padding='same', input_shape=INPUT_SHAPE))
model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2)))

# add flatten layer
model.add(tf.keras.layers.Flatten())


# add dense layers with some dropout
model.add(tf.keras.layers.Dense(512, activation='relu',kernel_regularizer = 'l2'))
model.add(tf.keras.layers.Dropout(rate=0.3))
model.add(tf.keras.layers.Dense(256, activation='relu',kernel_regularizer = 'l2'))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Dropout(rate=0.2))
model.add(tf.keras.layers.Dense(128, activation='relu'))

#model.add(tf.keras.layers.Flatten())


# add output layer
model.add(tf.keras.layers.Dense(5, activation='softmax'))

# compile model
model.compile(optimizer=adam, 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

# view model layers
model.summary()
EPOCHS = 100

es_callback = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, 
                                               restore_best_weights=True,
                                               verbose=2)

history = model.fit(train_generator,
                    validation_data = val_generator,
                    batch_size=16,
                    callbacks=[es_callback], 
                    epochs=EPOCHS,
                    verbose=1)
plotaccuracy(history)
plotloss(history)
preds = model.predict(data_test_normalized)
#if using softmax activation on output layer
predicted_labels  = np.argmax(preds,axis=1)

predicted_labels
#Accessing classnames with predicted clases
final_predictions = [class_names[i] for i in predicted_labels]
final_predictions<jupyter_output><empty_output><jupyter_text>**VGG19 Model**<jupyter_code># define input shape
INPUT_SHAPE_VGG = (256, 256, 3)

# get the VGG19 model
vgg_layers = tf.keras.applications.vgg19.VGG19(weights='imagenet', include_top=False, 
                                               input_shape=INPUT_SHAPE_VGG)

vgg_layers.summary()
# Fine-tune all the layers
for layer in vgg_layers.layers:
    layer.trainable = True

# Check the trainable status of the individual layers
for layer in vgg_layers.layers:
    print(layer, layer.trainable)
# define sequential model
model_vgg = tf.keras.models.Sequential()

# Add the vgg convolutional base model
model_vgg.add(vgg_layers)

# add flatten layer
model_vgg.add(tf.keras.layers.Flatten())


model_vgg.add(tf.keras.layers.Dense(512, activation='relu',input_dim=512,kernel_regularizer = 'l2'))
model_vgg.add(Dropout(.3))
model.add(tf.keras.layers.BatchNormalization())
model_vgg.add(Dense(256,activation=('relu'),kernel_regularizer = 'l1'))
model_vgg.add(Dropout(.2))

# add output layer
model_vgg.add(tf.keras.layers.Dense(5, activation='softmax'))

# compile model
model_vgg.compile(optimizer=sgd, 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

# view model layers
model_vgg.summary()
EPOCHS_VGG = 100

es_callback_VGG = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, 
                                               restore_best_weights=True,
                                               verbose=2)

history_VGG = model_vgg.fit(train_generator,
                    validation_data = val_generator,
                    batch_size=32,
                    callbacks=[es_callback_VGG], 
                    epochs=EPOCHS_VGG,
                    verbose=1)
plotaccuracy(history_VGG)
plotloss(history_VGG)
preds_VGG = model_vgg.predict(data_test_normalized)
#if using softmax activation on output layer
predicted_labels_VGG  = np.argmax(preds_VGG,axis=1)

predicted_labels_VGG
#Accessing classnames with predicted clases
final_predictions_VGG = [class_names[i] for i in predicted_labels_VGG]
final_predictions_VGG<jupyter_output><empty_output><jupyter_text>**InceptionV3 Model**<jupyter_code># define input shape
INPUT_SHAPE_INCEPTION = (256, 256, 3)

# get the Inception model
inception_layers = tf.keras.applications.inception_v3.InceptionV3(weights='imagenet', include_top=False, 
                                               input_shape=INPUT_SHAPE_INCEPTION)

inception_layers.summary()
# Fine-tune all the layers
for layer in inception_layers.layers:
    layer.trainable = False

# Check the trainable status of the individual layers
for layer in inception_layers.layers:
    print(layer, layer.trainable)
# define sequential model
model_inception = tf.keras.models.Sequential()

# Add the Inception convolutional base model
model_inception.add(inception_layers)

# add flatten layer
model_inception.add(tf.keras.layers.Flatten())


model_inception.add(tf.keras.layers.Dense(512, activation='relu',input_dim=512,kernel_regularizer = 'l2'))
model_inception.add(Dropout(.3))
model_inception.add(tf.keras.layers.BatchNormalization())
model_inception.add(Dense(256,activation=('relu'),kernel_regularizer = 'l1'))
model_inception.add(Dropout(.2))

# add output layer
model_inception.add(tf.keras.layers.Dense(5, activation='softmax'))

# compile model
model_inception.compile(optimizer=sgd, 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

# view model layers
model_inception.summary()
EPOCHS_INCEPTION = 100

es_callback_INCEPTION = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, 
                                               restore_best_weights=True,
                                               verbose=2)

history_INCEPTION = model_inception.fit(train_generator,
                    validation_data = val_generator,
                    batch_size=32,
                    callbacks=[es_callback_INCEPTION], 
                    epochs=EPOCHS_INCEPTION,
                    verbose=1)
plotaccuracy(history_INCEPTION)
plotloss(history_INCEPTION)
preds_INCEPTION = model_inception.predict(data_test_normalized)
#if using softmax activation on output layer
predicted_labels_INCEPTION  = np.argmax(preds_INCEPTION,axis=1)

predicted_labels_INCEPTION
#Accessing classnames with predicted clases
final_predictions_INCEPTION = [class_names[i] for i in predicted_labels_INCEPTION]
final_predictions_INCEPTION
!pip install -U efficientnet<jupyter_output>Requirement already up-to-date: efficientnet in /usr/local/lib/python3.6/dist-packages (1.1.0)
Requirement already satisfied, skipping upgrade: keras-applications<=1.0.8,>=1.0.7 in /usr/local/lib/python3.6/dist-packages (from efficientnet) (1.0.8)
Requirement already satisfied, skipping upgrade: scikit-image in /usr/local/lib/python3.6/dist-packages (from efficientnet) (0.16.2)
Requirement already satisfied, skipping upgrade: h5py in /usr/local/lib/python3.6/dist-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (2.10.0)
Requirement already satisfied, skipping upgrade: numpy>=1.9.1 in /usr/local/lib/python3.6/dist-packages (from keras-applications<=1.0.8,>=1.0.7->efficientnet) (1.18.5)
Requirement already satisfied, skipping upgrade: matplotlib!=3.0.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from scikit-image->efficientnet) (3.2.2)
Requirement already satisfied, skipping upgrade: imageio>=2.3.0 in /usr/local/lib/python3.6/dist-packages (from scikit-image->effici[...]<jupyter_text>**ResNet50 Model**<jupyter_code># define input shape
INPUT_SHAPE_RESNET = (256, 256, 3)

# get the Resnet model
ResNet_layers = tf.keras.applications.ResNet50(weights='imagenet', include_top=False, 
                                               input_shape=INPUT_SHAPE_RESNET)

ResNet_layers.summary()
# Fine-tune all the layers
for layer in ResNet_layers.layers:
    layer.trainable = True

# Check the trainable status of the individual layers
for layer in ResNet_layers.layers:
    print(layer, layer.trainable)
# define sequential model
model_ResNet = tf.keras.models.Sequential()

# Add the efficientNet convolutional base model
model_ResNet.add(ResNet_layers)

# add flatten layer
model_ResNet.add(tf.keras.layers.Flatten())


model_ResNet.add(tf.keras.layers.Dense(512, activation='relu',input_dim=512,kernel_regularizer = 'l2'))
model_ResNet.add(Dropout(.3))
model_ResNet.add(tf.keras.layers.BatchNormalization())
model_ResNet.add(Dense(256,activation=('relu'),kernel_regularizer = 'l1'))
model_ResNet.add(Dropout(.2))

# add output layer
model_ResNet.add(tf.keras.layers.Dense(5, activation='softmax'))

# compile model
model_ResNet.compile(optimizer=sgd, 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

# view model layers
model_ResNet.summary()
EPOCHS_RESNET = 50

es_callback_RESNET = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, 
                                               restore_best_weights=True,
                                               verbose=2)

history_RESNET = model_ResNet.fit(train_generator,
                    validation_data = val_generator,
                    batch_size=32,
                    callbacks=[es_callback_RESNET], 
                    epochs=EPOCHS_RESNET,
                    verbose=1)
plotaccuracy(history_RESNET)
plotloss(history_RESNET)
preds_RESNET = model_ResNet.predict(data_test_normalized)
#if using softmax activation on output layer
predicted_labels_RESNET  = np.argmax(preds_RESNET,axis=1)

predicted_labels_RESNET
#Accessing classnames with predicted clases
final_predictions_RESNET = [class_names[i] for i in predicted_labels_RESNET]
final_predictions_RESNET<jupyter_output><empty_output><jupyter_text>**Xception Model***This model gave the maximum accuracy for this particular dataset (and also by using proper regularization techniques like dropouts, BatchNormalization, L2 and L1 Regularization )*<jupyter_code># define input shape
INPUT_SHAPE_XCEPTION = (256, 256, 3)

# get the Resnet model
Xception_layers = tf.keras.applications.Xception(weights='imagenet', include_top=False, 
                                               input_shape=INPUT_SHAPE_XCEPTION)

Xception_layers.summary()
# Fine-tune all the layers
for layer in Xception_layers.layers:
    layer.trainable = True

# Check the trainable status of the individual layers
for layer in Xception_layers.layers:
    print(layer, layer.trainable)
# define sequential model
model_Xception = tf.keras.models.Sequential()

# Add the efficientNet convolutional base model
model_Xception.add(Xception_layers)

# add flatten layer
model_Xception.add(tf.keras.layers.Flatten())


model_Xception.add(tf.keras.layers.Dense(512, activation='relu',input_dim=512,kernel_regularizer = 'l2'))
model_Xception.add(Dropout(.3))
model_Xception.add(tf.keras.layers.BatchNormalization())
model_Xception.add(Dense(256,activation=('relu'),kernel_regularizer = 'l1'))
model_Xception.add(Dropout(.2))

# add output layer
model_Xception.add(tf.keras.layers.Dense(5, activation='softmax'))

# compile model
model_Xception.compile(optimizer=sgd, 
              loss='categorical_crossentropy', 
              metrics=['accuracy'])

# view model layers
model_Xception.summary()
EPOCHS_XCEPTION = 50

es_callback_XCEPTION = tf.keras.callbacks.EarlyStopping(monitor='val_loss', patience=3, 
                                               restore_best_weights=True,
                                               verbose=2)

history_XCEPTION = model_Xception.fit(train_generator,
                    validation_data = val_generator,
                    batch_size=32,
                    callbacks=[es_callback_XCEPTION], 
                    epochs=EPOCHS_XCEPTION,
                    verbose=1)
plotaccuracy(history_XCEPTION)
plotloss(history_XCEPTION)
preds_XCEPTION = model_Xception.predict(data_test_normalized)
#if using softmax activation on output layer
predicted_labels_XCEPTION  = np.argmax(preds_XCEPTION,axis=1)

predicted_labels_XCEPTION
#Accessing classnames with predicted clases
final_predictions_XCEPTION = [class_names[i] for i in predicted_labels_XCEPTION]
final_predictions_XCEPTION<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09646191646191646, 'fraction_numerical': 0.018378378378378378, 'mean_word_length': 3.600135623869801, 'pattern_counts': {'":': 0, '<': 51, '<?xml version=': 0, '>': 58, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 58, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '235256', 'license': 'MIT', 'n_tokens_mistral': 6457, 'n_tokens_neox': 6111, 'n_words': 1383, 'path': 'Recognize_Animals.ipynb', 'repo_name': 'jkcg-learning/100DaysOfMLCode', 'size': '20362'}