<jupyter_start><jupyter_text>Image Classification using Convolutional Neural Networks in PyTorch  Introduction The objective of this notebook is to gain an insight into object recognition using a popular deep learning algorithm called **Convolutional Neural Network(CNN)**. **PyTorch** is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab. It is a free and open-source software released under the Modified BSD license.The dataset that is used for this task is the **CIFAR-10 Dataset** which is an established computer-vision dataset used for object recognition. It is a subset of the 80 million tiny images dataset and consists of 60,000 32x32 color images containing one of 10 object classes, with 6000 images per class. It was collected by <NAME>, <NAME>, and <NAME>. It is available for download at Kaggle.  Dataset The CIFAR-10 data consists of 60,000 32x32 color images in 10 classes, with 6000 images per class. There are 50,000 training images and 10,000 test images in the official data. We have preserved the train/test split from the original dataset.  The provided files are:train.7z - a folder containing the training images in png formattest.7z - a folder containing the test images in png formattrainLabels.csv - the training labels The label classes in the dataset are:airplane automobile bird cat deer dog frog horse ship truck  Importing required libraries<jupyter_code>import os
import torch
import torchvision
import tarfile
from torchvision.datasets.utils import download_url
from torch.utils.data import random_split
project_name='05-cifar10-cnn'<jupyter_output><empty_output><jupyter_text>We'll download the images in PNG format from https://course.fast.ai/datasets, using some helper functions from the **torchvision** and **tarfile** packages.<jupyter_code># Downloading the dataset
dataset_url = "https://s3.amazonaws.com/fast-ai-imageclas/cifar10.tgz"
download_url(dataset_url, '.')
# Extract from archive
with tarfile.open('./cifar10.tgz', 'r:gz') as tar:
    tar.extractall(path='./data')<jupyter_output><empty_output><jupyter_text>The dataset is extracted to the directory **data/cifar10**. It contains 2 folders **train** and **test**, containing the training set (50000 images) and test set (10000 images) respectively. Each of them contains **10 folders**, one for each class of images. Let's verify this using **os.listdir**.<jupyter_code>data_dir = './data/cifar10'

print(os.listdir(data_dir))
classes = os.listdir(data_dir + "/train")
print(classes)
airplane_files = os.listdir(data_dir + "/train/airplane")
print('No. of training examples for airplanes:', len(airplane_files))
print(airplane_files[:5])
ship_test_files = os.listdir(data_dir + "/test/ship")
print("No. of test examples for ship:", len(ship_test_files))
print(ship_test_files[:5])<jupyter_output>No. of test examples for ship: 1000
['0001.png', '0002.png', '0003.png', '0004.png', '0005.png']<jupyter_text>We can use the **ImageFolder** class from **torchvision** to load the data as **PyTorch tensors**.<jupyter_code>from torchvision.datasets import ImageFolder
from torchvision.transforms import ToTensor
dataset = ImageFolder(data_dir+'/train', transform=ToTensor())<jupyter_output><empty_output><jupyter_text>Let's look at a sample element from the training dataset. Each element is a tuple, containing a **image tensor** and a **label**. Since the data consists of **32x32 px color images with 3 channels (RGB)**, each image tensor has the shape **(3, 32, 32)**.<jupyter_code>img, label = dataset[0]
print(img.shape, label)
img<jupyter_output>torch.Size([3, 32, 32]) 0<jupyter_text>The list of classes is stored in the **.classes** property of the dataset. The numeric label for each element corresponds to index of the element's label in the list of classes.<jupyter_code>print(dataset.classes)<jupyter_output>['airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship', 'truck']<jupyter_text>We can view the image using **matplotlib**, but we need to change the tensor dimensions to (32,32,3). Let's create a helper function to display an image and its label.<jupyter_code>import matplotlib
import matplotlib.pyplot as plt
%matplotlib inline

matplotlib.rcParams['figure.facecolor'] = '#ffffff'   # (#ffffff) stands for white. It is the color code of white
def show_example(img, label):
    print('Label: ', dataset.classes[label], "("+str(label)+")")
    plt.imshow(img.permute(1, 2, 0))<jupyter_output><empty_output><jupyter_text>Let's look at a couple of images from the dataset. As you can tell, the 32x32px images are quite difficult to identify, even for the human eye. We can try changing the indices below to view different images.<jupyter_code>show_example(*dataset[0])
show_example(*dataset[1099])<jupyter_output>Label:  airplane (0)<jupyter_text>Train/Test/Validation Data split While building real world machine learning models, it is very common to split the dataset into 3 parts:**Training set** - used to train the model i.e. compute the loss and adjust the weights of the model using gradient descent.**Validation set** - used to evaluate the model while training, adjust hyperparameters (learning rate etc.) and pick the best version of the model.**Test set** - used to compare different models, or different types of modeling approaches, and report the final accuracy of the model.Since there's no predefined validation set, we can set aside a small portion (5000 images) of the training set to be used as the validation set. We'll use the **random_split** helper method from **PyTorch** to do this. To ensure that we always create the same validation set, we'll also set a seed for the random number generator.<jupyter_code>random_seed = 42
torch.manual_seed(random_seed);
val_size = 5000
train_size = len(dataset) - val_size

train_ds, val_ds = random_split(dataset, [train_size, val_size])
len(train_ds), len(val_ds)<jupyter_output><empty_output><jupyter_text>We can now create data loaders for training and validation, to load the data in **batches**.<jupyter_code>from torch.utils.data.dataloader import DataLoader

batch_size=128
train_dl = DataLoader(train_ds, batch_size, shuffle=True, num_workers=4, pin_memory=True)
val_dl = DataLoader(val_ds, batch_size*2, num_workers=4, pin_memory=True)<jupyter_output><empty_output><jupyter_text>We can look at batches of images from the dataset using the **make_grid** method from **torchvision**. Each time the following code is run, we get a different batch, since the sampler shuffles the indices before creating batches.<jupyter_code>from torchvision.utils import make_grid

def show_batch(dl):
    for images, labels in dl:
        fig, ax = plt.subplots(figsize=(12, 6))
        ax.set_xticks([]); ax.set_yticks([])
        ax.imshow(make_grid(images, nrow=16).permute(1, 2, 0))
        break
show_batch(train_dl)
#Importing required packages for implementing CNN in Pytorch
import torch.nn as nn
import torch.nn.functional as F<jupyter_output><empty_output><jupyter_text>Before we define the entire model, let's look at how a single convolutional layer followed by a max-pooling layer operates on the data.<jupyter_code>simple_model = nn.Sequential(
    nn.Conv2d(3, 8, kernel_size=3, stride=1, padding=1),
    nn.MaxPool2d(2, 2)
)
for images, labels in train_dl:
    print('images.shape:', images.shape)
    out = simple_model(images)
    print('out.shape:', out.shape)
    break<jupyter_output>images.shape: torch.Size([128, 3, 32, 32])
out.shape: torch.Size([128, 8, 16, 16])<jupyter_text>The **Conv2d** layer transforms a **3-channel image** to a **8-channel feature map**, and the **MaxPool2d** layer **halves the height and width**. The feature map gets smaller as we add more layers, until we are finally left with a small feature map, which can be flattened into a **vector**. We can then add some fully connected neural network layers at the end to get vector of size 10 for each image. Let's define the model by extending an **ImageClassificationBase** class which contains helper methods for training & validation.<jupyter_code>class ImageClassificationBase(nn.Module):
    def training_step(self, batch):
        images, labels = batch 
        out = self(images)                  # Generate predictions
        loss = F.cross_entropy(out, labels) # Calculate loss
        return loss
    
    def validation_step(self, batch):
        images, labels = batch 
        out = self(images)                    # Generate predictions
        loss = F.cross_entropy(out, labels)   # Calculate loss
        acc = accuracy(out, labels)           # Calculate accuracy
        return {'val_loss': loss.detach(), 'val_acc': acc}
        
    def validation_epoch_end(self, outputs):
        batch_losses = [x['val_loss'] for x in outputs]
        epoch_loss = torch.stack(batch_losses).mean()   # Combine losses
        batch_accs = [x['val_acc'] for x in outputs]
        epoch_acc = torch.stack(batch_accs).mean()      # Combine accuracies
        return {'val_loss': epoch_loss.item(), 'val_acc': epoch_acc.item()}
    
    def epoch_end(self, epoch, result):
        print("Epoch [{}], train_loss: {:.4f}, val_loss: {:.4f}, val_acc: {:.4f}".format(
            epoch, result['train_loss'], result['val_loss'], result['val_acc']))
        
def accuracy(outputs, labels):
    _, preds = torch.max(outputs, dim=1)
    return torch.tensor(torch.sum(preds == labels).item() / len(preds))<jupyter_output><empty_output><jupyter_text>We'll use **nn.Sequential** to chain the layers and activations functions into a single network architecture.<jupyter_code>#Defining the Convolutional Neural Network(CNN) model
class Cifar10CnnModel(ImageClassificationBase):
    def __init__(self):
        super().__init__()
        self.network = nn.Sequential(
            nn.Conv2d(3, 32, kernel_size=4, padding=2),
            nn.ReLU(),
            nn.Conv2d(32, 64, kernel_size=8, stride=2, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(2, 2), # output: 64 x 16 x 16

            nn.Conv2d(64, 128, kernel_size=2, stride=2, padding=4),
            nn.ReLU(),
            nn.Conv2d(128, 128, kernel_size=1, stride=1, padding=6),
            nn.ReLU(),
            nn.MaxPool2d(2, 2), # output: 128 x 8 x 8

            nn.Conv2d(128, 256, kernel_size=4, stride=1, padding=2),
            nn.ReLU(),
            nn.Conv2d(256, 256, kernel_size=2, stride=2, padding=4),
            nn.ReLU(),
            nn.MaxPool2d(2, 2), # output: 256 x 4 x 4

            nn.Flatten(), 
            nn.Linear(256*4*4, 1024),
            nn.ReLU(),
            nn.Linear(1024, 512),
            nn.ReLU(),
            nn.Linear(512, 10))
        
    def forward(self, xb):
        return self.network(xb)
model = Cifar10CnnModel()
model<jupyter_output><empty_output><jupyter_text>Let's verify that the model produces the expected output on a batch of training data. The 10 outputs for each image can be interpreted as probabilities for the 10 target classes (after applying **softmax**), and the class with the highest probability is chosen as the label predicted by the model for the input image.<jupyter_code>for images, labels in train_dl:
    print("images.shape: ", images.shape)
    out=model(images)
    print("out.shape: ", out.shape)
    print("out[0]: ", out[0])
    break
probs=F.softmax(out[0], dim=0)
torch.sum(probs), probs<jupyter_output><empty_output><jupyter_text>the tensor above represents the **probabilities of the 10 classes**, i.e., 'airplane', 'automobile', 'bird', 'cat', 'deer', 'dog', 'frog', 'horse', 'ship' and 'truck'. As we can see, the probability of the first class predicted by the model is the highest which is 'airplane'.  Let us check if it is actually true.<jupyter_code>show_example(images[0], labels[0])<jupyter_output>Label:  frog (tensor(6))<jupyter_text>The actual label of the image is **'frog'**. And it is quite obvious from the image too. But our model predicted it as **'airplane'**. This clearly indicates that our model is **not trained yet**. That is why the model is just generating random predictions. Let's train the model. To seamlessly use a GPU, if one is available, we define a couple of helper functions (**get_default_device** and **to_device**) and a helper class **DeviceDataLoader** to move our model & data to the GPU as required.<jupyter_code>def get_default_device():
    """Pick GPU if available, else CPU"""
    if torch.cuda.is_available():
        return torch.device('cuda')
    else:
        return torch.device('cpu')
    
def to_device(data, device):
    """Move tensor(s) to chosen device"""
    if isinstance(data, (list,tuple)):
        return [to_device(x, device) for x in data]
    return data.to(device, non_blocking=True)

class DeviceDataLoader():
    """Wrap a dataloader to move data to a device"""
    def __init__(self, dl, device):
        self.dl = dl
        self.device = device
        
    def __iter__(self):
        """Yield a batch of data after moving it to device"""
        for b in self.dl: 
            yield to_device(b, self.device)

    def __len__(self):
        """Number of batches"""
        return len(self.dl)
device = get_default_device()
device<jupyter_output><empty_output><jupyter_text>We can now wrap our training and validation data loaders using **DeviceDataLoader** for automatically transferring batches of data to the GPU, and use to_device to move our model to the GPU.<jupyter_code>train_dl = DeviceDataLoader(train_dl, device)
val_dl = DeviceDataLoader(val_dl, device)
to_device(model, device);<jupyter_output><empty_output><jupyter_text>Training the Model We'll define two functions: **fit** and **evaluate** to train the model using **gradient descent** and evaluate its performance on the **validation set**.<jupyter_code>@torch.no_grad()
def evaluate(model, val_loader):
    model.eval()
    outputs = [model.validation_step(batch) for batch in val_loader]
    return model.validation_epoch_end(outputs)

def fit(epochs, lr, model, train_loader, val_loader, opt_func=torch.optim.SGD):
    history = []
    optimizer = opt_func(model.parameters(), lr)
    for epoch in range(epochs):
        # Training Phase 
        model.train()
        train_losses = []
        for batch in train_loader:
            loss = model.training_step(batch)
            train_losses.append(loss)
            loss.backward()
            optimizer.step()
            optimizer.zero_grad()
        # Validation phase
        result = evaluate(model, val_loader)
        result['train_loss'] = torch.stack(train_losses).mean().item()
        model.epoch_end(epoch, result)
        history.append(result)
    return history<jupyter_output><empty_output><jupyter_text>Before we begin training, let's instantiate the model once again and see how it performs on the validation set with the initial set of parameters.<jupyter_code>model = to_device(Cifar10CnnModel(), device)
evaluate(model, val_dl)<jupyter_output><empty_output><jupyter_text>The initial accuracy is **around 10%**, which is what one might expect from a randomly intialized model (since it has a 1 in 10 chance of getting a label right by guessing randomly).We'll use the following hyperparmeters (**learning rate, no. of epochs, batch_size etc.**) to train our model.<jupyter_code>num_epochs = 10
opt_func = torch.optim.Adam
lr = 0.001
history = fit(num_epochs, lr, model, train_dl, val_dl, opt_func)<jupyter_output>Epoch [0], train_loss: 1.8061, val_loss: 1.4997, val_acc: 0.4352
Epoch [1], train_loss: 1.2957, val_loss: 1.1415, val_acc: 0.5773
Epoch [2], train_loss: 1.0485, val_loss: 0.9922, val_acc: 0.6437
Epoch [3], train_loss: 0.8746, val_loss: 0.8393, val_acc: 0.7051
Epoch [4], train_loss: 0.7243, val_loss: 0.8364, val_acc: 0.7113
Epoch [5], train_loss: 0.6166, val_loss: 0.7580, val_acc: 0.7442
Epoch [6], train_loss: 0.5247, val_loss: 0.7239, val_acc: 0.7577
Epoch [7], train_loss: 0.4461, val_loss: 0.7314, val_acc: 0.7683
Epoch [8], train_loss: 0.3632, val_loss: 0.7275, val_acc: 0.7767
Epoch [9], train_loss: 0.2941, val_loss: 0.7555, val_acc: 0.7812<jupyter_text>As we can see, the model has achieved an accuracy of 78% after training for 10 epochs which is quite reasonable. We can also plot the valdation set accuracies to study how the model improves over time.<jupyter_code>def plot_accuracies(history):
    accuracies = [x['val_acc'] for x in history]
    plt.plot(accuracies, '-x')
    plt.xlabel('epoch')
    plt.ylabel('accuracy')
    plt.title('Accuracy vs. No. of epochs');
plot_accuracies(history)<jupyter_output><empty_output><jupyter_text>The model reaches an accuracy of around **78%**, and by looking at the graph, it seems unlikely that the model will achieve an accuracy higher than 80% even after training for a long time. This suggests that we might need to use a more powerful model to capture the relationship between the images and the labels more accurately. This can be done by **adding more convolutional layers to our model, or incrasing the no. of channels in each convolutional layer, or by using regularization techniques**.We can also plot the **training** and **validation losses** to study the trend.<jupyter_code>def plot_losses(history):
    train_losses = [x.get('train_loss') for x in history]
    val_losses = [x['val_loss'] for x in history]
    plt.plot(train_losses, '-bx')
    plt.plot(val_losses, '-rx')
    plt.xlabel('epoch')
    plt.ylabel('loss')
    plt.legend(['Training', 'Validation'])
    plt.title('Loss vs. No. of epochs');
plot_losses(history)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10621513495238633, 'fraction_numerical': 0.027722995435848313, 'mean_word_length': 4.224609949955844, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 89, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '451541', 'license': 'Apache-2.0', 'n_tokens_mistral': 5745, 'n_tokens_neox': 5327, 'n_words': 1898, 'path': 'Image Classification on CIFAR-10 dataset using CNN in Pytorch.ipynb', 'repo_name': 'abhinavtembulkar/Image-Classification-Using-CNN-in-Pytorch', 'size': '17769'}