<jupyter_start><jupyter_text>| Name | Description | Date| :- |-------------: | :-:|<NAME>| Convolutional Neural Networks - 5th  | Finalized on 23rd of August 2019 | width="750" align="center"> Convolutional Neural Networks- CIFAR-10 image classification with CNN  - Last time, we have attempted to classify images in the CIFAR-10 dataset with simple CNN having one convolutional & pooling layers. The final result was accuracy score of 0.645.  - Again, let's try to improve the classification performance by implementing deeper CNN with more layers<jupyter_code>!pip3 install torch torchvision
import numpy as np
import pandas as pd
import torch, torchvision
import torch.nn as nn
import torch.nn.functional as F
torch.__version__<jupyter_output><empty_output><jupyter_text>1. Import & process dataset- _CIFAR10_ dataset can be downloaded by ```torchvision```  - [torchvision.datasets](https://pytorch.org/docs/stable/torchvision/datasets.html)<jupyter_code>from torchvision import datasets
import torchvision.transforms as transforms

train_dataset = datasets.CIFAR10(root = "/", train = True, download = True, transform = transforms.ToTensor())
test_dataset = datasets.CIFAR10(root = "/", train = False, download = True, transform = transforms.ToTensor())<jupyter_output>0it [00:00, ?it/s]<jupyter_text>2. Creating CNN model and training- Create and train deeper CNN model, having two convolutional & pooling layers![](https://www.researchgate.net/profile/Qianzhou_Du2/publication/322477802/figure/fig3/AS:582461356511232@1515881017676/Illustration-of-Convolutional-Neural-Network-CNN-Architecture.png)<jupyter_code># create data loaders 
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size = 128, shuffle = True)
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size = 128, shuffle = False)
# create CNN with one convolution/pooling layer
class net(nn.Module):
  def __init__(self, input_dim, num_filters, conv_kernel_size, pool_kernel_size, stride, padding, num_classes):
    super(net, self).__init__()
    self.input_dim = input_dim
    conv_output_size = int((input_dim - conv_kernel_size + 2 * padding)/stride) + 1   # first conv layer output size
    pool_output_size = int((conv_output_size - pool_kernel_size)/stride) + 1          # first pooling layer output size
    conv_output_size = int((pool_output_size - conv_kernel_size + 2 * padding)/stride) + 1   # second conv layer output size
    pool_output_size = int((conv_output_size - pool_kernel_size)/stride) + 1          # second pooling layer output size
    
    self.conv1 = nn.Conv2d(3, num_filters[0], kernel_size = conv_kernel_size, stride = stride, padding = padding)     
    self.pool1 = nn.MaxPool2d(kernel_size = pool_kernel_size, stride = stride)
    self.conv2 = nn.Conv2d(num_filters[0], num_filters[1], kernel_size = conv_kernel_size, stride = stride, padding = padding)     
    self.pool2 = nn.MaxPool2d(kernel_size = pool_kernel_size, stride = stride)
    self.relu = nn.ReLU()
    self.dense = nn.Linear(pool_output_size * pool_output_size * num_filters[-1], num_classes)     
    
  def forward(self, x):
    x = self.conv1(x)
    x = self.relu(x)
    x = self.pool1(x)
    x = self.conv2(x)
    x = self.relu(x)
    x = self.pool2(x)
    x = x.view(x.size(0), -1)   # resize to fit into final dense layer
    x = self.dense(x)
    return x
# hyperparameters
DEVICE = torch.device('cuda')
INPUT_DIM = 32
NUM_FILTERS = (32, 32)
CONV_KERNEL_SIZE = 3
POOL_KERNEL_SIZE = 3
STRIDE = 1
PADDING = 1
NUM_CLASSES = 10
LEARNING_RATE = 1e-3
NUM_EPOCHS = 30              
model = net(INPUT_DIM, NUM_FILTERS, CONV_KERNEL_SIZE, POOL_KERNEL_SIZE, STRIDE, PADDING, NUM_CLASSES).to(DEVICE)
criterion = nn.CrossEntropyLoss()   # do not need softmax layer when using CEloss criterion
optimizer = torch.optim.Adam(model.parameters(), lr = LEARNING_RATE)
# training for NUM_EPOCHS
for i in range(NUM_EPOCHS):
  temp_loss = []
  for (x, y) in train_loader:
    x, y = x.float().to(DEVICE), y.to(DEVICE)
    outputs = model(x)
    loss = criterion(outputs, y)
    temp_loss.append(loss.item())
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
  print("Loss at {}th epoch: {}".format(i, np.mean(temp_loss)))<jupyter_output>Loss at 0th epoch: 1.4810623134798406
Loss at 1th epoch: 1.2012358184360787
Loss at 2th epoch: 1.1048996875353176
Loss at 3th epoch: 1.0310378291112992
Loss at 4th epoch: 0.9728409978739746
Loss at 5th epoch: 0.9245658644934749
Loss at 6th epoch: 0.8730855590242255
Loss at 7th epoch: 0.8399296389211475
Loss at 8th epoch: 0.8048995495452296
Loss at 9th epoch: 0.7791642356101814
Loss at 10th epoch: 0.7552188227853507
Loss at 11th epoch: 0.7366858548520471
Loss at 12th epoch: 0.7143801430911969
Loss at 13th epoch: 0.6967860703425639
Loss at 14th epoch: 0.6731867749825158
Loss at 15th epoch: 0.6541185029174971
Loss at 16th epoch: 0.6448789071244048
Loss at 17th epoch: 0.6236151499516519
Loss at 18th epoch: 0.6079566321714455
Loss at 19th epoch: 0.58996188198514
Loss at 20th epoch: 0.5754009889977058
Loss at 21th epoch: 0.5623295599847193
Loss at 22th epoch: 0.5469853096758314
Loss at 23th epoch: 0.5329701231263787
Loss at 24th epoch: 0.5172345991939535
Loss at 25th epoch: 0.503110246127828[...]<jupyter_text>3. Evaluation- Evaluate the trained CNN model with accuracy score   - Store probability of each instance to a list and compare it with true y label<jupyter_code>y_pred, y_true = [], []
with torch.no_grad():
  for x, y in test_loader:
    x, y = x.float().to(DEVICE), y.to(DEVICE)
    outputs = F.softmax(model(x)).max(1)[-1]       # predicted label
    y_true += list(y.cpu().numpy())                # true label
    y_pred += list(outputs.cpu().numpy())   
# evaluation result
from sklearn.metrics import accuracy_score
accuracy_score(y_true, y_pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10337155570768441, 'fraction_numerical': 0.10302926578812253, 'mean_word_length': 4.4362790697674415, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 17, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '592538', 'license': 'MIT', 'n_tokens_mistral': 2442, 'n_tokens_neox': 1990, 'n_words': 543, 'path': 'PyTorch Exercise/Deep Learning with PyTorch/DL-with-pytorch - 5 [CNN].ipynb', 'repo_name': 'rezapci/PyTorch', 'size': '5849'}