<jupyter_start><jupyter_text>First contact with PyTorchWe will first train a network to classify MNIST to get familiarized with PyTorch's syntax.First, we define our neural network. In PyTorch,  NN are just Python classes that inherit from nn.Module. We declare all the layers that we are going to use in the constructor (init method), and then define how we should do the forward pass in the `forward` method. The `backward` method is done automatically, so we do not need to code it.<jupyter_code>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(in_channels=1, out_channels=20, kernel_size=5, stride=1)
        self.conv2 = nn.Conv2d(in_channels=20, out_channels=50, kernel_size=5, stride=1)
        self.fc1 = nn.Linear(in_features=4*4*50, out_features=500)
        self.fc2 = nn.Linear(in_features=500, out_features=10)

    def forward(self, x):
        x = F.relu(self.conv1(x))
        x = F.max_pool2d(x, 2, 2)
        x = F.relu(self.conv2(x))
        x = F.max_pool2d(x, 2, 2)
        x = x.view(x.size(0), -1)  # Flatten while keeping the batch dimension
        x = F.relu(self.fc1(x))
        x = F.log_softmax(self.fc2(x), dim=1)
        return x
   

model = Net().to(device)  # Move model to GPU<jupyter_output><empty_output><jupyter_text>We now define the dataloaders that will feed the data to our NN during trainig in batches. The `dataset` class `torchvision.dataset.MNIST` already has the MNIST dataset loaded. We could add data augmentation here.<jupyter_code>train_dataset = datasets.MNIST("data/MNIST", train=True, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),  # Here we could add extra data augmentation
                       transforms.Normalize((0.1307,), (0.3081,)),
                   ]))
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=100, shuffle=True)

test_dataset = datasets.MNIST("data/MNIST", train=False, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),
                       transforms.Normalize((0.1307,), (0.3081,)),
                   ]))
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=100, shuffle=False)<jupyter_output>0it [00:00, ?it/s]<jupyter_text>For our optimizer, we will use SGD, and it will optimize the parameters of our model. For our loss, we wil use Negative Log Likelihood (together with the log_softmax of the last layer it is equivalent to the Cross Entropy while being more numerically stable).<jupyter_code>optimizer = optim.SGD(model.parameters(), lr=0.1)
criterion = nn.NLLLoss()<jupyter_output><empty_output><jupyter_text>We will define a custom metric to monitor during training.<jupyter_code>def accuracy(preds, y):
    preds = preds.argmax(dim=1, keepdim=True) # get the index of the max log-probability
    correct = preds.eq(y.view_as(preds)).sum().item()
    return correct/preds.size(0)<jupyter_output><empty_output><jupyter_text>Here, we do our training. At each step we do a forward pass of our input, we compute the gradients and we do an optimizer step. At the end of each epoch we also compute metrics with our validation dataset.<jupyter_code>epochs = 3
for epoch in range(epochs):
    model.train()  # Train mode
    for i, (x, y) in enumerate(train_loader):
        x, y = x.to(device), y.to(device)
        optimizer.zero_grad()  # Reset gradients from last step
        preds = model(x)  # Forward pass
        loss = criterion(preds, y)
        loss.backward()  # Compute gradients
        optimizer.step()  # Do an optimizer step
        if i%20 == 0:
            print(f"TRAIN Epoch {epoch}: [{i}/{len(train_loader)}] Loss: {loss.item():.4f}; Accuracy: {accuracy(preds, y)}")
        
    
    model.eval()  # Test mode
    for i, (x, y) in enumerate(test_loader):
        x, y = x.to(device), y.to(device)
        preds = model(x)
        loss = criterion(preds, y)
        if i%20 == 0:
            print(f"TEST Epoch {epoch}: [{i}/{len(test_loader)}] Loss: {loss.item():.4f}; Accuracy: {accuracy(preds, y)}")<jupyter_output>TRAIN Epoch 0: [0/600] Loss: 2.3067; Accuracy: 0.05
TRAIN Epoch 0: [20/600] Loss: 0.8959; Accuracy: 0.82
TRAIN Epoch 0: [40/600] Loss: 0.3778; Accuracy: 0.91
TRAIN Epoch 0: [60/600] Loss: 0.1984; Accuracy: 0.95
TRAIN Epoch 0: [80/600] Loss: 0.4193; Accuracy: 0.86
TRAIN Epoch 0: [100/600] Loss: 0.4728; Accuracy: 0.86
TRAIN Epoch 0: [120/600] Loss: 0.1536; Accuracy: 0.94
TRAIN Epoch 0: [140/600] Loss: 0.1265; Accuracy: 0.98
TRAIN Epoch 0: [160/600] Loss: 0.2830; Accuracy: 0.9
TRAIN Epoch 0: [180/600] Loss: 0.1418; Accuracy: 0.97
TRAIN Epoch 0: [200/600] Loss: 0.0996; Accuracy: 0.97
TRAIN Epoch 0: [220/600] Loss: 0.1225; Accuracy: 0.97
TRAIN Epoch 0: [240/600] Loss: 0.1065; Accuracy: 0.97
TRAIN Epoch 0: [260/600] Loss: 0.0996; Accuracy: 0.96
TRAIN Epoch 0: [280/600] Loss: 0.0800; Accuracy: 0.99
TRAIN Epoch 0: [300/600] Loss: 0.1069; Accuracy: 0.96
TRAIN Epoch 0: [320/600] Loss: 0.1939; Accuracy: 0.97
TRAIN Epoch 0: [340/600] Loss: 0.1037; Accuracy: 0.97
TRAIN Epoch 0: [360/600] Loss: 0.15[...]<jupyter_text>Exercice 1: train with CIFAR10You should complete the following code to train a network with CIFAR10.If you want, you can try to add data augmentation, dropout and weight decay to regularize the model.<jupyter_code>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = 
        self.conv2 = 
        self.fc1 = 
        self.fc2 = 

    def forward(self, x):

        return x
   

model = Net().to(device)


train_dataset = datasets.CIFAR10("data/CIFAR10", train=True, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),  # Here we could add extra data augmentation
                       transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                   ]))
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=100, shuffle=True)

test_dataset = datasets.CIFAR10("data/CIFAR10", train=False, download=True,
                   transform=transforms.Compose([
                       transforms.ToTensor(),
                       transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5)),
                   ]))
test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=100, shuffle=False)

optimizer = optim.SGD(model.parameters(), lr=0.1)
criterion = nn.NLLLoss()

epochs = 3
for epoch in range(epochs):
    model.train()  # Train mode
    for i, (x, y) in enumerate(train_loader):
        x, y = x.to(device), y.to(device)
        optimizer.zero_grad()  # Reset gradients from last step
        preds = model(x)  # Forward pass
        loss = criterion(preds, y)
        loss.backward()  # Compute gradients
        optimizer.step()  # Do an optimizer step
        if i%20 == 0:
            print(f"TRAIN Epoch {epoch}: [{i}/{len(train_loader)}] Loss: {loss.item():.4f}; Accuracy: {accuracy(preds, y)}")
        
    
    model.eval()  # Test mode
    for i, (x, y) in enumerate(test_loader):
        x, y = x.to(device), y.to(device)
        preds = model(x)
        loss = criterion(preds, y)
        if i%20 == 0:
            print(f"TEST Epoch {epoch}: [{i}/{len(test_loader)}] Loss: {loss.item():.4f}; Accuracy: {accuracy(preds, y)}")<jupyter_output><empty_output><jupyter_text>Pretrained imagenet models In both PyTorch and Keras there are many models pretrained with Imagenet. You can see the lists here: https://pytorch.org/docs/stable/torchvision/models.html and https://keras.io/applications/.We will see how to use a pretrained VGG16 from PyTorch to predict images taken from the internet. We first declare our model with `pretrained=True`, which will download the pretrained weights.<jupyter_code>import torchvision.models as models

model = models.vgg16(pretrained=True)<jupyter_output><empty_output><jupyter_text>We also load the 1000 imagenet labels in a Python dictionary.<jupyter_code>import json
import ast
urllib.request.urlretrieve("https://gist.githubusercontent.com/yrevar/942d3a0ac09ec9e5eb3a/raw/238f720ff059c1f82f368259d1ca4ffa5dd8f9f5/imagenet1000_clsidx_to_labels.txt", "labels.json")
with open("labels.json") as f:
    labels = ast.literal_eval(f.read())
print(labels)<jupyter_output>{0: 'tench, Tinca tinca', 1: 'goldfish, Carassius auratus', 2: 'great white shark, white shark, man-eater, man-eating shark, Carcharodon carcharias', 3: 'tiger shark, Galeocerdo cuvieri', 4: 'hammerhead, hammerhead shark', 5: 'electric ray, crampfish, numbfish, torpedo', 6: 'stingray', 7: 'cock', 8: 'hen', 9: 'ostrich, Struthio camelus', 10: 'brambling, Fringilla montifringilla', 11: 'goldfinch, Carduelis carduelis', 12: 'house finch, linnet, Carpodacus mexicanus', 13: 'junco, snowbird', 14: 'indigo bunting, indigo finch, indigo bird, Passerina cyanea', 15: 'robin, American robin, Turdus migratorius', 16: 'bulbul', 17: 'jay', 18: 'magpie', 19: 'chickadee', 20: 'water ouzel, dipper', 21: 'kite', 22: 'bald eagle, American eagle, Haliaeetus leucocephalus', 23: 'vulture', 24: 'great grey owl, great gray owl, Strix nebulosa', 25: 'European fire salamander, Salamandra salamandra', 26: 'common newt, Triturus vulgaris', 27: 'eft', 28: 'spotted salamander, Ambystoma maculatum', 29: 'axolotl, mu[...]<jupyter_text>We can now download a sample image to test our network.<jupyter_code>import urllib
from PIL import Image
urllib.request.urlretrieve("https://3c1703fe8d.site.internapcdn.net/newman/gfx/news/hires/2018/2-dog.jpg", "image.jpg")
%ls
pil_image = Image.open("image.jpg")
imshow(np.asarray(pil_image))<jupyter_output>[0m[01;34mdata[0m/  image.jpg  labels.json  [01;34mMask_RCNN[0m/  [01;34msample_data[0m/<jupyter_text>We should do some preprocessing before feeding the image to our network. The following transformations are the same that were used when training with Imagenet. Note that we have to add a batch dimension at the beginning, even if we are feeding just one image to the network.<jupyter_code>preprocess = transforms.Compose([transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])
torch_image = preprocess(pil_image)
torch_image = torch_image.unsqueeze(0)  # Add batch dimension
print("Image shape: ", torch_image.shape)
print("Image values: ", torch_image)<jupyter_output>Image shape:  torch.Size([1, 3, 224, 224])
Image values:  tensor([[[[-1.0733, -1.0562, -1.0048,  ...,  0.2453,  0.2967,  0.2967],
          [-1.0562, -1.0390, -1.0048,  ...,  0.1939,  0.2453,  0.2624],
          [-1.0219, -0.9877, -0.9534,  ...,  0.1597,  0.2111,  0.2453],
          ...,
          [-1.4843, -1.4843, -1.4672,  ...,  1.8208,  1.8893,  1.9064],
          [-1.4843, -1.4843, -1.4672,  ...,  1.8550,  1.8722,  1.9235],
          [-1.4843, -1.4843, -1.4672,  ...,  1.8379,  1.8893,  1.9407]],

         [[-0.8102, -0.7752, -0.7402,  ...,  0.4853,  0.5203,  0.5378],
          [-0.7752, -0.7577, -0.7227,  ...,  0.4503,  0.4853,  0.5203],
          [-0.7577, -0.7227, -0.7052,  ...,  0.4153,  0.4678,  0.5028],
          ...,
          [-1.1779, -1.1779, -1.1779,  ...,  0.9755,  1.0805,  1.2031],
          [-1.1779, -1.1954, -1.1779,  ...,  1.0105,  1.0630,  1.2031],
          [-1.1779, -1.1779, -1.1779,  ...,  1.0280,  1.1155,  1.2381]],

         [[-1.6127, -1.5953, -1.5604,  ..., [...]<jupyter_text>Now we can get the predictions of our network. Note that a network trained with imagenet will have 1000 outputs. Here we only show the top 3 predictions and their probabilities.<jupyter_code>output = model(torch_image)
print("Output shape: ", output.shape)
probs, idxs = output.topk(3)
print(probs[0])
print([labels[idx.item()] for idx in idxs[0]])<jupyter_output>Output shape:  torch.Size([1, 1000])
tensor([14.8598, 13.9637, 10.4510], grad_fn=<SelectBackward>)
['golden retriever', 'Labrador retriever', 'soccer ball']<jupyter_text>Exercice 2Do the prediction with other images using different networks and compare the results.  You can use the function following function predict_from_url. The list of pretrained PyTorch networks can be found here https://pytorch.org/docs/stable/torchvision/models.html<jupyter_code>def predict_from_url(url, model):
    urllib.request.urlretrieve(url, "image.jpg")
    pil_image = Image.open("image.jpg")
    imshow(np.asarray(pil_image))
    torch_image = preprocess(pil_image).unsqueeze(0)
    output = model(torch_image)
    probs, idxs = output.topk(3)
    print(probs[0])
    print([labels[idx.item()] for idx in idxs[0]])

predict_from_url("https://img.purch.com/h/1400/aHR0cDovL3d3dy5saXZlc2NpZW5jZS5jb20vaW1hZ2VzL2kvMDAwLzEwMi8xNjIvb3JpZ2luYWwvYmVlci1hbmQtd2hlYXQuanBn", model)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13882592042523256, 'fraction_numerical': 0.07902759321503948, 'mean_word_length': 3.453184824225548, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 6, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '267102', 'license': 'MIT', 'n_tokens_mistral': 5290, 'n_tokens_neox': 4631, 'n_words': 1211, 'path': 'Upc/Computer_Vision_Days_6_9_13_17/aidl2019_cv_lab5_imagenet.ipynb', 'repo_name': 'igorfyago/Andrew-Ng-Machine-Learning', 'size': '12793'}