<jupyter_start><jupyter_text>Transfer LearningMost of the time you won't want to train a whole convolutional network yourself. Modern ConvNets training on huge datasets like ImageNet take weeks on multiple GPUs. > Instead, most people use a pretrained network either as a fixed feature extractor, or as an initial network to fine tune. We do transfer learning when we use a pre-trained network on images not in the training set. We'll use transfer learning to train a network that can classify cats and dogs photos.In this notebook, you'll be using a pre-trained model from the [ImageNet dataset](http://www.image-net.org/) as a feature extractor. Below is a diagram showing the architecture of the model we'll be using. It has a series of convolutional and maxpooling layers, and some fully-connected layers at the end that aid in classifying the images (For us it's cat's and dogs).The idea is to keep all the convolutional layers, but **replace the final fully-connected layer** with our own classifier. This way we can use VGGNet as a _fixed feature extractor_ for our images then easily train a simple classifier on top of that. * Use all but the last fully-connected layer as a fixed feature extractor.* Define a new, final classification layer and apply it to a task of our choice!<jupyter_code>%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import matplotlib.pyplot as plt

import torch
from torch import nn
from torch import optim
from torch.autograd import Variable
import torch.nn.functional as F
from torchvision import datasets, transforms, models<jupyter_output><empty_output><jupyter_text>Most of the require a 224x224 image as input.We need to resize the images.<jupyter_code>data_dir = 'dogs_vs_cats'

# The model takes 224x224 images as input, so we resize all of them
data_transform = transforms.Compose([transforms.RandomResizedCrop(224), 
                                      transforms.ToTensor()])

train_data = datasets.ImageFolder(data_dir + '/train', transform=data_transform)
test_data = datasets.ImageFolder(data_dir + '/test', transform=data_transform)

train_loader = torch.utils.data.DataLoader(train_data, batch_size=64, shuffle=True)
test_loader = torch.utils.data.DataLoader(test_data, batch_size=32)


# The model has two parts: the features, and the classifier. 

classifier = nn.Sequential(
    nn.Linear(2048, 512),
    nn.ReLU(),
    nn.Dropout(p=0.2),
    nn.Linear(512, 2),
    nn.LogSoftmax(dim=1))

# replace the classifier with our own
model.fc = classifier

# use negative log likelihood loss
criterion = nn.NLLLoss()
optimizer = optim.Adam(model.fc.parameters(), lr=0.003)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model.to(device)
epochs = 1
steps = 0
running_loss = 0
print_every = 5

for epoch in range(epochs):
    for images, labels in train_loader:
        steps += 1
        
        images, labels = images.to(device), labels.to(device)
        
        # clear gradients
        optimizer.zero_grad()
        
        logps = model(images)
        loss = criterion(logps, labels)
        loss.backward()
        optimizer.step()
        
        running_loss += loss.item()
        
        # after training loop, test our network's accuracy and loss
        if steps % print_every == 0:
            model.eval() # turn model into evaluation mode to make predictions
            test_loss = 0
            accuracy = 0
            
            for images, labels in test_loader:
                
                images, labels = images.to(device), labels.to(device)
                
                logps = model(images)
                loss = criterion(logps, labels)
                test_loss += loss.item()
                
                # accuracy
                ps = torch.exp(logps)
                top_ps, top_class = ps.topk(1, dim=1)
                equality = top_class == labels.view(*top_class.shape)
                accuracy += torch.mean(equality.type(torch.FloatTensor)).item()
                
                
                print(f'Epoch {epoch+1}/{epochs}.. ')
                print(f'Train loss: {running_loss/print_every}')
                print(f'Test loss: {test_loss/len(test_loader)}')
                print(f'Test accuracy: {accuracy}/{len(test_loader)}')
                
                running_loss=0
                model.train()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08164657721173528, 'fraction_numerical': 0.011143961792130999, 'mean_word_length': 2.746166950596252, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 10, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '41639', 'license': 'MIT', 'n_tokens_mistral': 1271, 'n_tokens_neox': 1195, 'n_words': 430, 'path': 'intro-notebooks/transfer_learning.ipynb', 'repo_name': 'gitgik/pytorch', 'size': '4397'}