<jupyter_start><jupyter_text>**Building a strong image classification model from less data**The implementation is a slight variation of the one in https://gist.github.com/fchollet/0830affa1f7f19fd47b06d4cf89ed44dMainly, in this kernel , the method flow(x,y) is used whereas, in the above gist, method flow_from_directory(directory) is used.For more info, you can refer https://keras.io/preprocessing/image/The change is made to have an appropriate kernel to deal with the way data is structured in kaggle. Appropriate changes in other parts of the source code is also done. **Perform the necessary imports.**<jupyter_code>import os, cv2, re, random
import numpy as np
import pandas as pd
from keras.preprocessing.image import ImageDataGenerator
from keras.preprocessing.image import img_to_array, load_img
from keras import layers, models, optimizers
from keras import backend as K
from sklearn.model_selection import train_test_split<jupyter_output>Using TensorFlow backend.<jupyter_text>**Data dimensions and paths**<jupyter_code>img_width = 150
img_height = 150
TRAIN_DIR = '../input/train/'
TEST_DIR = '../input/test/'
train_images_dogs_cats = [TRAIN_DIR+i for i in os.listdir(TRAIN_DIR)] # use this for full dataset
test_images_dogs_cats = [TEST_DIR+i for i in os.listdir(TEST_DIR)]<jupyter_output><empty_output><jupyter_text>**Helper function to sort the image files based on the numeric value in each file name.**<jupyter_code>def atoi(text):
    return int(text) if text.isdigit() else text

def natural_keys(text):
    return [ atoi(c) for c in re.split('(\d+)', text) ]<jupyter_output><empty_output><jupyter_text>**Sort the traning set. Use 1300 images each of cats and dogs instead of all 25000 to speed up the learning process.****Sort the test set**<jupyter_code>train_images_dogs_cats.sort(key=natural_keys)
train_images_dogs_cats = train_images_dogs_cats[0:1300] + train_images_dogs_cats[12500:13800] 

test_images_dogs_cats.sort(key=natural_keys)<jupyter_output><empty_output><jupyter_text>**Now the images have to be represented in numbers. For this, using the openCV library read and resize the image.  ****Generate labels for the supervised learning set.****Below is the helper function to do so.**<jupyter_code>def prepare_data(list_of_images):
    """
    Returns two arrays: 
        x is an array of resized images
        y is an array of labels
    """
    x = [] # images as arrays
    y = [] # labels
    
    for image in list_of_images:
        x.append(cv2.resize(cv2.imread(image), (img_width,img_height), interpolation=cv2.INTER_CUBIC))
    
    for i in list_of_images:
        if 'dog' in i:
            y.append(1)
        elif 'cat' in i:
            y.append(0)
        #else:
            #print('neither cat nor dog name present in images')
            
    return x, y<jupyter_output><empty_output><jupyter_text>**Generate X and Y using the helper function above****Since K.image_data_format() is channel_last,  input_shape to the first keras layer will be (img_width, img_height, 3). '3' since it is a color image**<jupyter_code>X, Y = prepare_data(train_images_dogs_cats)
print(K.image_data_format())<jupyter_output>channels_last<jupyter_text>**Split the data set containing 2600 images into 2 parts, training set and validation set. Later, you will see that accuracy and loss on the validation set will also be reported while fitting the model using training set.**<jupyter_code># First split the data in two sets, 80% for training, 20% for Val/Test)
X_train, X_val, Y_train, Y_val = train_test_split(X,Y, test_size=0.2, random_state=1)
nb_train_samples = len(X_train)
nb_validation_samples = len(X_val)
batch_size = 16<jupyter_output><empty_output><jupyter_text>**We will be using the Sequential model from Keras to form the Neural Network. Sequential Model is  used to construct simple models with linear stack of layers. ****More info on Sequential model and Keras in general at https://keras.io/getting-started/sequential-model-guide/ and https://github.com/keras-team/keras**<jupyter_code>model = models.Sequential()

model.add(layers.Conv2D(32, (3, 3), input_shape=(img_width, img_height, 3)))
model.add(layers.Activation('relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2)))

model.add(layers.Conv2D(32, (3, 3)))
model.add(layers.Activation('relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2)))

model.add(layers.Conv2D(64, (3, 3)))
model.add(layers.Activation('relu'))
model.add(layers.MaxPooling2D(pool_size=(2, 2)))

model.add(layers.Flatten())
model.add(layers.Dense(64))
model.add(layers.Activation('relu'))
model.add(layers.Dropout(0.5))
model.add(layers.Dense(1))
model.add(layers.Activation('sigmoid'))

model.compile(loss='binary_crossentropy',
              optimizer='rmsprop',
              metrics=['accuracy'])

model.summary()<jupyter_output>_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 148, 148, 32)      896       
_________________________________________________________________
activation_1 (Activation)    (None, 148, 148, 32)      0         
_________________________________________________________________
max_pooling2d_1 (MaxPooling2 (None, 74, 74, 32)        0         
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 72, 72, 32)        9248      
_________________________________________________________________
activation_2 (Activation)    (None, 72, 72, 32)        0         
_________________________________________________________________
max_pooling2d_2 (MaxPooling2 (None, 36, 36, 32)        0         
_________________________________________________________________
conv2d_3 ([...]<jupyter_text>**This is the augmentation configuration we will use for training and validation**<jupyter_code>train_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

val_datagen = ImageDataGenerator(
    rescale=1. / 255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)<jupyter_output><empty_output><jupyter_text>**Prepare generators for training and validation sets**<jupyter_code>train_generator = train_datagen.flow(np.array(X_train), Y_train, batch_size=batch_size)
validation_generator = val_datagen.flow(np.array(X_val), Y_val, batch_size=batch_size)<jupyter_output><empty_output><jupyter_text>**Start training the model!****For better accuracy and lower loss, we are using an epoch of 30. Epoch value can be increased for better results.**<jupyter_code>history = model.fit_generator(
    train_generator, 
    steps_per_epoch=nb_train_samples // batch_size,
    epochs=30,
    validation_data=validation_generator,
    validation_steps=nb_validation_samples // batch_size
)<jupyter_output>Epoch 1/30
130/130 [==============================] - 70s 537ms/step - loss: 0.7205 - acc: 0.5144 - val_loss: 0.6863 - val_acc: 0.5840
Epoch 2/30
130/130 [==============================] - 69s 531ms/step - loss: 0.6861 - acc: 0.5832 - val_loss: 0.6626 - val_acc: 0.6369
Epoch 3/30
130/130 [==============================] - 66s 508ms/step - loss: 0.6500 - acc: 0.6312 - val_loss: 0.6165 - val_acc: 0.6667
Epoch 4/30
130/130 [==============================] - 67s 515ms/step - loss: 0.6067 - acc: 0.6813 - val_loss: 0.5966 - val_acc: 0.6607
Epoch 5/30
130/130 [==============================] - 66s 509ms/step - loss: 0.5821 - acc: 0.7010 - val_loss: 0.6239 - val_acc: 0.6865
Epoch 6/30
130/130 [==============================] - 66s 512ms/step - loss: 0.5760 - acc: 0.7144 - val_loss: 0.5725 - val_acc: 0.7103
Epoch 7/30
130/130 [==============================] - 68s 522ms/step - loss: 0.5524 - acc: 0.7279 - val_loss: 0.5304 - val_acc: 0.7540
Epoch 8/30
130/130 [==============================] - 6[...]<jupyter_text>**Well done for a small training set and a small Epoch! Accuracy on the training set is close to 84% while on the validation set is close to 79%** **Saving the model in Keras is simple as this! ** **It is quite helpful for reuse.**<jupyter_code>model.save_weights('model_wieghts.h5')
model.save('model_keras.h5')<jupyter_output><empty_output><jupyter_text>**Time to predict classification using the model on the test set.****Generate X_test and Y_test**<jupyter_code>X_test, Y_test = prepare_data(test_images_dogs_cats) #Y_test in this case will be []<jupyter_output><empty_output><jupyter_text>**This is the augmentation configuration we will use for testing. Only rescaling.**<jupyter_code>test_datagen = ImageDataGenerator(rescale=1. / 255)<jupyter_output><empty_output><jupyter_text>**Prepare generator for test set and start predicting on it.**<jupyter_code>test_generator = val_datagen.flow(np.array(X_test), batch_size=batch_size)
prediction_probabilities = model.predict_generator(test_generator, verbose=1)<jupyter_output>782/782 [==============================] - 158s 202ms/step<jupyter_text>**Generate .csv for submission**<jupyter_code>counter = range(1, len(test_images_dogs_cats) + 1)
solution = pd.DataFrame({"id": counter, "label":list(prediction_probabilities)})
cols = ['label']

for col in cols:
    solution[col] = solution[col].map(lambda x: str(x).lstrip('[').rstrip(']')).astype(float)

solution.to_csv("dogsVScats.csv", index = False)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1413089672758181, 'fraction_numerical': 0.04876753081172971, 'mean_word_length': 5.156311314584696, 'pattern_counts': {'":': 2, '<': 60, '<?xml version=': 0, '>': 60, 'https://': 4, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '551410', 'license': 'Apache-2.0', 'n_tokens_mistral': 3343, 'n_tokens_neox': 2985, 'n_words': 787, 'path': 'Dogs_vs_Cats_Classification/Kernel/dogs_vs_cats_notebook.ipynb', 'repo_name': 'mankutimma/machine-learning-kaggle', 'size': '9412'}