<jupyter_start><jupyter_text>Transfer Learning with Resnet In this notebook we load a small datasets that contains dolphins and elephants. We classify the images using CNNs and compare two approaches and see what worsk better:1. Training a CNN from sratch against2. Finetuning a pretrained ResNet.<jupyter_code>import torch
from torchvision import transforms
import torchvision
import torch.nn as nn
import torch.nn.functional as F
import torchvision.datasets as datasets

torch.manual_seed(0)<jupyter_output><empty_output><jupyter_text>Let's load our data a have a look at the shape of some images:<jupyter_code>dataset = datasets.ImageFolder(root='../data/anmials')
for i, data in enumerate(dataset):
    print(data)
    if i == 10:
        break<jupyter_output>(<PIL.Image.Image image mode=RGB size=300x179 at 0x7FDD242741F0>, 0)
(<PIL.Image.Image image mode=RGB size=300x179 at 0x7FDD2427FA90>, 0)
(<PIL.Image.Image image mode=RGB size=300x166 at 0x7FDD2437E8E0>, 0)
(<PIL.Image.Image image mode=RGB size=300x259 at 0x7FDD242741F0>, 0)
(<PIL.Image.Image image mode=RGB size=300x225 at 0x7FDD2437E280>, 0)
(<PIL.Image.Image image mode=RGB size=300x277 at 0x7FDD2427FA90>, 0)
(<PIL.Image.Image image mode=RGB size=300x183 at 0x7FDD2437EEE0>, 0)
(<PIL.Image.Image image mode=RGB size=300x225 at 0x7FDD2427F670>, 0)
(<PIL.Image.Image image mode=RGB size=300x214 at 0x7FDD2437E8E0>, 0)
(<PIL.Image.Image image mode=RGB size=300x223 at 0x7FDD2427FA90>, 0)
(<PIL.Image.Image image mode=RGB size=300x277 at 0x7FDD2437E280>, 0)<jupyter_text>We see that the pictures have all width=300 but a varying height. To use them in transfer learning they need to have the standard shape (224, 224), which is the data format of ImageNet (on which most pretrained models are trained on).   To get them into this shape, we first increase the height to 224 (this will also increase the height) and then take the 224 square which is center in the middle.<jupyter_code>image_transforms = transforms.Compose([
             transforms.Resize(size=224),
             transforms.CenterCrop(size=224),
             transforms.ToTensor(),
             transforms.Normalize([0.485, 0.456, 0.406],[0.229, 0.224, 0.225]) # standard normalization for transfer learning
    ])
data = datasets.ImageFolder(root='../data/anmials', transform=image_transforms)

print(len(data), "data points")<jupyter_output>129 data points<jupyter_text>Next, we split the data into train and test and define the data loaders.<jupyter_code>train_set, test_set = torch.utils.data.random_split(data, [100, 29])

batch_size = 10
trainloader = torch.utils.data.DataLoader(train_set, batch_size=batch_size,
                                          shuffle=True)
testloader = torch.utils.data.DataLoader(test_set, batch_size=29,
                                         shuffle=False)<jupyter_output><empty_output><jupyter_text>Tasks: Task 1.Train a CNN from scratch to identify the object on the image (dolphin or elephant). For this, use the same CNN architecture as in notebook `5_CNN_CIFAR10.ipynb` (to make this work, you need to adjust some network parameters for this dataset). Train for 20 epochs on the train data and afterwards compute accuracy on the test data.<jupyter_code>class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(44944, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 2)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = torch.flatten(x, 1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x
import torch.optim as optim

net = Net()

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

for epoch in range(20):
    
    loss_batch = 0
    for i, data in enumerate(trainloader, 0):
        
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        loss_batch += loss.item()
        
    print(f'{epoch + 1}, loss: {loss_batch:.5f}')
    loss_batch = 0

print('Finished Training')
correct = 0
total = 0

wrong_images = []
wrong_labesl = []

with torch.no_grad():
    for data in testloader:
        images, labels = data
        # calculate outputs by running images through the network
        outputs = net(images)
        # the class with the highest energy is what we choose as prediction
        predicted = torch.argmax(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print(f'Accuracy of the network on the test images: {100 * correct // total} %')<jupyter_output>Accuracy of the network on the test images: 86 %<jupyter_text>Task 2:Instead of training a CNN, load a pretrained ResNet18 and only train the last layer (see the lecture slides how that works). Train again for 20 epochs and compare the results.<jupyter_code>model = torchvision.models.resnet18(pretrained=True)

for param in model.parameters():
      param.requires_grad = False

fc_inputs = model.fc.in_features
model.fc = nn.Linear(fc_inputs, 2)
import torch.optim as optim

net = model

criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)

for epoch in range(20):

    loss_batch = 0.0
    for i, data in enumerate(trainloader):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        #print(labels)

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        
        #print(outputs.shape)
        #print(outputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        loss_batch += loss.item()
        
    print(f'{epoch + 1}, loss: {loss_batch:.5f}')
    loss_batch = 0

print('Finished Training')
correct = 0
total = 0

wrong_images = []
wrong_labesl = []

with torch.no_grad():
    for data in testloader:
        images, labels = data
        outputs = net(images)
        predicted = torch.argmax(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print(f'Accuracy of the network on the test images: {100 * correct // total} %')<jupyter_output>Accuracy of the network on the test images: 100 %
{'fraction_non_alphanumeric': 0.10388830505930041, 'fraction_numerical': 0.04939198318570785, 'mean_word_length': 3.3119741100323625, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '630907', 'license': 'MIT', 'n_tokens_mistral': 2367, 'n_tokens_neox': 2107, 'n_words': 652, 'path': 'solutions/6_TransferLearning_ResNet-Solution.ipynb', 'repo_name': 'pabair/mml2-ss22', 'size': '6661'}