<jupyter_start><jupyter_text>Pytorch to fastai, Bridging the Gap
> Understanding how to bring Pytorch code into the fastai space with minimal headache

* toc: true
* badges: true
* comments: true<jupyter_code>#hide_input
from wwf.utils import state_versions
state_versions(['fastai', 'fastcore', 'torch', 'torchvision'])<jupyter_output><empty_output><jupyter_text>Addressing the Elephant in the Room

I recently posted a [tweet](https://twitter.com/TheZachMueller/status/1359584461873111044?s=20) asking about what people struggle with the most in `fastai`, and the resounding answer was how to integrate *minimally* with Pytorch. An impression seems to have been made that to use `fastai` you *must* use the complete `fastai` API only, and nothing else.

Let's clear up that misconception now:

> Important: `fastai` at its core is a *training* loop, designed to be framework agnostic. You can use any flavor of Pytorch you want, and only use `fastai` to quickly and effictively train a model with state-of-the-art practices  The Plan

Now that the misconceptions have been addressed, let's walk through just how that is going to happen. We're going to follow the official Pytorch [CIFAR10](https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html) tutorial and show what needs to minimally happen in the `fastai` framework to take full advantage of the `Learner`. This will include:

* The `Dataset`
* The `DataLoaders`
* The model
* The optimizer  The Dataset and DataLoaders

Following from the tutorial, we're going to load in the dataset using only torchvision. First we'll grab our imports:<jupyter_code>import torch
import torchvision
import torchvision.transforms as transforms<jupyter_output><empty_output><jupyter_text>Next we're going to definine some minimal transforms:<jupyter_code>transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5,0.5,0.5), (0.5,0.5,0.5))])<jupyter_output><empty_output><jupyter_text>Before downloading our train and test sets:

> Note: I'm using naming conventions similar to how `fastai` names things, so you can see how these can relate to each other<jupyter_code>dset_train = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
dset_test = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)<jupyter_output>Files already downloaded and verified<jupyter_text>Next we'll make our `Dataloaders`:<jupyter_code>trainloader = torch.utils.data.DataLoader(dset_train, batch_size=4,
                                          shuffle=True, num_workers=2)
testloader = torch.utils.data.DataLoader(dset_test, batch_size=4,
                                         shuffle=False, num_workers=2)<jupyter_output><empty_output><jupyter_text>And that's as far as we'll go from there for now, let's move onto the model next  The Model

We'll bring in the architecture from the tutorial and use it here:<jupyter_code>import torch.nn as nn
import torch.nn.functional as F
class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x<jupyter_output><empty_output><jupyter_text>And finally we'll make an instance of it:<jupyter_code>net = Net()<jupyter_output><empty_output><jupyter_text>Loss Function and Optimizer

Next we'll bring in their loss function and optimizer. The loss function is simple enough:<jupyter_code>criterion = nn.CrossEntropyLoss()<jupyter_output><empty_output><jupyter_text>However the optimizer requires a *little* bit of `fastai` magic, specifically in the form of an `OptimWrapper`. Our optimizer function should be defined as below:<jupyter_code>from fastai.optimizer import OptimWrapper
from torch import optim
def opt_func(params, **kwargs): return OptimWrapper(optim.SGD(params, lr=0.001))<jupyter_output><empty_output><jupyter_text>Training

Now we have everything needed to train a model, so now let's bring in `fastai`'s training loop, also known as the `Learner`. 

`fastai`'s `Learner` expects `DataLoaders` to be used, rather than simply one `DataLoader`, so let's make that:

> Note: fastai also expects a validation `DataLoader` to be present, so we'll be tying the `testloader` in here<jupyter_code>from fastai.data.core import DataLoaders
dls = DataLoaders(trainloader, testloader)<jupyter_output><empty_output><jupyter_text>Finally we're going to wrap it all up in a `Learner`. As mentioned before, the `Learner` is the glue that merges everything together and enables users to utilize Leslie Smith's One-Cycle Policy, the learning rate finder, and other `fastai` training goodies. 

Let's make it by passing in our `dls`, the model, the optimizer, and the loss function:<jupyter_code>from fastai.learner import Learner<jupyter_output><empty_output><jupyter_text>To get `fastai`'s fancy-looking progress bar, we need to import the `ProgressCallback`:<jupyter_code>from fastai.callback.progress import ProgressCallback<jupyter_output><empty_output><jupyter_text>We also need to pass in the `CudaCallback` so our batches can be pushed to the GPU (`fastai`'s `DataLoaders` can do this automatically)<jupyter_code>from fastai.callback.data import CudaCallback
learn = Learner(dls, net, loss_func=criterion, opt_func=opt_func, cbs=[CudaCallback])<jupyter_output><empty_output><jupyter_text>Finally, let's do some minimal training.  Now we have everything needed to do a basic `fit`:
> Note: Since we already passed in a learning rate to `Learner` we don't need to pass one in here<jupyter_code>learn.fit(2)<jupyter_output><empty_output><jupyter_text>What's Next?

Great, so now we've trained our model, but what do we do with it? How do I get it out?

Your model lives in `learn.model`, and we've already seen that we passed in a regular Pytorch model earlier. Since we're using fastai's base `Learner` class, the model itself was untouched. As a result, it's *still* a regular Pytorch model we can save away:<jupyter_code>torch.save(learn.model.state_dict(), './cifar_net.pth')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09244712990936556, 'fraction_numerical': 0.014048338368580061, 'mean_word_length': 4.596787827557058, 'pattern_counts': {'":': 0, '<': 60, '<?xml version=': 0, '>': 65, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '162883', 'license': 'Apache-2.0', 'n_tokens_mistral': 2167, 'n_tokens_neox': 2058, 'n_words': 695, 'path': '_notebooks/2021-02-14-Pytorchtofastai.ipynb', 'repo_name': 'jss367/fastblog', 'size': '6620'}