title image classification from scratch fchollethttps twitter comfchollet date created 20200427 last modified 20221110 description training an image classifier from scratch on the kaggle cats vs dogs dataset accelerator gpu introduction this example shows how to do image classification from scratch starting from jpeg image files on disk without leveraging pretrained weights or a premade keras application model we demonstrate the workflow on the kaggle cats vs dogs binary classification dataset we use the imagedatasetfromdirectory utility to generate the datasets and we use keras image preprocessing layers for image standardization and data augmentation setup load the data the cats vs dogs dataset raw data download first let s download the 786m zip archive of the raw data now we have a petimages folder which contain two subfolders cat and dog each subfolder contains image files for each category filter out corrupted images when working with lots of realworld image data corrupted images are a common occurence let s filter out badlyencoded images that do not feature the string jfif in their header delete corrupted image generate a dataset visualize the data here are the first 9 images in the training dataset as you can see label 1 is dog and label 0 is cat using image data augmentation when you don t have a large image dataset it s a good practice to artificially introduce sample diversity by applying random yet realistic transformations to the training images such as random horizontal flipping or small random rotations this helps expose the model to different aspects of the training data while slowing down overfitting let s visualize what the augmented samples look like by applying dataaugmentation repeatedly to the first image in the dataset standardizing the data our image are already in a standard size 180x180 as they are being yielded as contiguous float32 batches by our dataset however their rgb channel values are in the 0 255 range this is not ideal for a neural network in general you should seek to make your input values small here we will standardize values to be in the 0 1 by using a rescaling layer at the start of our model two options to preprocess the data there are two ways you could be using the dataaugmentation preprocessor option 1 make it part of the model like this python inputs keras inputshapeinputshape x dataaugmentationinputs x layers rescaling1 255x rest of the model with this option your data augmentation will happen on device synchronously with the rest of the model execution meaning that it will benefit from gpu acceleration note that data augmentation is inactive at test time so the input samples will only be augmented during fit not when calling evaluate or predict if you re training on gpu this may be a good option option 2 apply it to the dataset so as to obtain a dataset that yields batches of augmented images like this python augmentedtrainds trainds map lambda x y dataaugmentationx trainingtrue y with this option your data augmentation will happen on cpu asynchronously and will be buffered before going into the model if you re training on cpu this is the better option since it makes data augmentation asynchronous and nonblocking in our case we ll go with the second option if you re not sure which one to pick this second option asynchronous preprocessing is always a solid choice configure the dataset for performance let s apply data augmentation to our training dataset and let s make sure to use buffered prefetching so we can yield data from disk without having io becoming blocking apply dataaugmentation to the training images prefetching samples in gpu memory helps maximize gpu utilization build a model we ll build a small version of the xception network we haven t particularly tried to optimize the architecture if you want to do a systematic search for the best model configuration consider using kerastunerhttps github comkerasteamkerastuner note that we start the model with the dataaugmentation preprocessor followed by a rescaling layer we include a dropout layer before the final classification layer entry block project residual train the model we get to 90 validation accuracy after training for 25 epochs on the full dataset in practice you can train for 50 epochs before validation performance starts degrading run inference on new data note that data augmentation and dropout are inactive at inference time introduction this example shows how to do image classification from scratch starting from jpeg image files on disk without leveraging pre trained weights or a pre made keras application model we demonstrate the workflow on the kaggle cats vs dogs binary classification dataset we use the image_dataset_from_directory utility to generate the datasets and we use keras image preprocessing layers for image standardization and data augmentation setup load the data the cats vs dogs dataset raw data download first let s download the 786m zip archive of the raw data now we have a petimages folder which contain two subfolders cat and dog each subfolder contains image files for each category filter out corrupted images when working with lots of real world image data corrupted images are a common occurence let s filter out badly encoded images that do not feature the string jfif in their header delete corrupted image generate a dataset visualize the data here are the first 9 images in the training dataset as you can see label 1 is dog and label 0 is cat using image data augmentation when you don t have a large image dataset it s a good practice to artificially introduce sample diversity by applying random yet realistic transformations to the training images such as random horizontal flipping or small random rotations this helps expose the model to different aspects of the training data while slowing down overfitting let s visualize what the augmented samples look like by applying data_augmentation repeatedly to the first image in the dataset standardizing the data our image are already in a standard size 180x180 as they are being yielded as contiguous float32 batches by our dataset however their rgb channel values are in the 0 255 range this is not ideal for a neural network in general you should seek to make your input values small here we will standardize values to be in the 0 1 by using a rescaling layer at the start of our model two options to preprocess the data there are two ways you could be using the data_augmentation preprocessor option 1 make it part of the model like this python inputs keras input shape input_shape x data_augmentation inputs x layers rescaling 1 255 x rest of the model with this option your data augmentation will happen on device synchronously with the rest of the model execution meaning that it will benefit from gpu acceleration note that data augmentation is inactive at test time so the input samples will only be augmented during fit not when calling evaluate or predict if you re training on gpu this may be a good option option 2 apply it to the dataset so as to obtain a dataset that yields batches of augmented images like this python augmented_train_ds train_ds map lambda x y data_augmentation x training true y with this option your data augmentation will happen on cpu asynchronously and will be buffered before going into the model if you re training on cpu this is the better option since it makes data augmentation asynchronous and non blocking in our case we ll go with the second option if you re not sure which one to pick this second option asynchronous preprocessing is always a solid choice configure the dataset for performance let s apply data augmentation to our training dataset and let s make sure to use buffered prefetching so we can yield data from disk without having i o becoming blocking apply data_augmentation to the training images prefetching samples in gpu memory helps maximize gpu utilization build a model we ll build a small version of the xception network we haven t particularly tried to optimize the architecture if you want to do a systematic search for the best model configuration consider using kerastuner https github com keras team keras tuner note that we start the model with the data_augmentation preprocessor followed by a rescaling layer we include a dropout layer before the final classification layer entry block set aside residual project residual add back residual set aside next residual train the model we get to 90 validation accuracy after training for 25 epochs on the full dataset in practice you can train for 50 epochs before validation performance starts degrading run inference on new data note that data augmentation and dropout are inactive at inference time create batch axis
import tensorflow as tf
import keras
from keras import layers
import os
from pathlib import Path
import matplotlib.pyplot as plt



fpath = keras.utils.get_file(
    origin="https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_5340.zip"
)
dirpath = Path(fpath).parent.absolute()
os.system(f"unzip -q {fpath} -d {dirpath}")



os.system(f"ls {dirpath}/PetImages/")



import os

num_skipped = 0
for folder_name in ("Cat", "Dog"):
    folder_path = os.path.join(dirpath, "PetImages", folder_name)
    for fname in os.listdir(folder_path):
        fpath = os.path.join(folder_path, fname)
        try:
            fobj = open(fpath, "rb")
            is_jfif = tf.compat.as_bytes("JFIF") in fobj.peek(10)
        finally:
            fobj.close()

        if not is_jfif:
            num_skipped += 1
            
            os.remove(fpath)

print(f"Deleted {num_skipped} images")



image_size = (180, 180)
batch_size = 128

train_ds, val_ds = keras.utils.image_dataset_from_directory(
    os.path.join(dirpath, "PetImages"),
    validation_split=0.2,
    subset="both",
    seed=1337,
    image_size=image_size,
    batch_size=batch_size,
)



plt.figure(figsize=(10, 10))
for images, labels in train_ds.take(1):
    for i in range(9):
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(images[i].numpy().astype("uint8"))
        plt.title(int(labels[i]))
        plt.axis("off")



data_augmentation = keras.Sequential(
    [
        layers.RandomFlip("horizontal"),
        layers.RandomRotation(0.1),
    ]
)



plt.figure(figsize=(10, 10))
for images, _ in train_ds.take(1):
    for i in range(9):
        augmented_images = data_augmentation(images)
        ax = plt.subplot(3, 3, i + 1)
        plt.imshow(augmented_images[0].numpy().astype("uint8"))
        plt.axis("off")







print("Make datasets")

train_ds = train_ds.map(
    lambda img, label: (data_augmentation(img), label),
    num_parallel_calls=tf.data.AUTOTUNE,
)

train_ds = train_ds.prefetch(tf.data.AUTOTUNE)
val_ds = val_ds.prefetch(tf.data.AUTOTUNE)




def make_model(input_shape, num_classes):
    inputs = keras.Input(shape=input_shape)

    
    x = layers.Rescaling(1.0 / 255)(inputs)
    x = layers.Conv2D(128, 3, strides=2, padding="same")(x)
    x = layers.BatchNormalization()(x)
    x = layers.Activation("relu")(x)

    previous_block_activation = x  

    for size in [256, 512, 728]:
        x = layers.Activation("relu")(x)
        x = layers.SeparableConv2D(size, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)

        x = layers.Activation("relu")(x)
        x = layers.SeparableConv2D(size, 3, padding="same")(x)
        x = layers.BatchNormalization()(x)

        x = layers.MaxPooling2D(3, strides=2, padding="same")(x)

        
        residual = layers.Conv2D(size, 1, strides=2, padding="same")(
            previous_block_activation
        )
        x = layers.add([x, residual])  
        previous_block_activation = x  

    x = layers.SeparableConv2D(1024, 3, padding="same")(x)
    x = layers.BatchNormalization()(x)
    x = layers.Activation("relu")(x)

    x = layers.GlobalAveragePooling2D()(x)
    if num_classes == 2:
        activation = "sigmoid"
        units = 1
    else:
        activation = "softmax"
        units = num_classes

    x = layers.Dropout(0.5)(x)
    outputs = layers.Dense(units, activation=activation)(x)
    return keras.Model(inputs, outputs)


model = make_model(input_shape=image_size + (3,), num_classes=2)
keras.utils.plot_model(model, show_shapes=True)



epochs = 25

callbacks = [
    keras.callbacks.ModelCheckpoint("save_at_{epoch}.keras"),
]

model.compile(
    optimizer=keras.optimizers.Adam(1e-3),
    loss="binary_crossentropy",
    metrics=["accuracy"],
)
model.fit(
    train_ds,
    epochs=epochs,
    callbacks=callbacks,
    validation_data=val_ds,
)





img = keras.utils.load_img(
    f"{dirpath}/PetImages/Cat/6779.jpg", target_size=image_size
)
img_array = keras.utils.img_to_array(img)
img_array = tf.expand_dims(img_array, 0)  

predictions = model.predict(img_array)
score = float(predictions[0])
print(f"This image is {100 * (1 - score):.2f}% cat and {100 * score:.2f}% dog.")