<jupyter_start><jupyter_text>Saving and loading multiple models in one file using PyTorch============================================================Saving and loading multiple models can be helpful for reusing modelsthat you have previously trained.Introduction------------When saving a model comprised of multiple ``torch.nn.Modules``, such asa GAN, a sequence-to-sequence model, or an ensemble of models, you mustsave a dictionary of each model’s state_dict and correspondingoptimizer. You can also save any other items that may aid you inresuming training by simply appending them to the dictionary.To load the models, first initialize the models and optimizers, thenload the dictionary locally using ``torch.load()``. From here, you caneasily access the saved items by simply querying the dictionary as youwould expect.In this recipe, we will demonstrate how to save multiple models to onefile using PyTorch.Setup-----Before we begin, we need to install ``torch`` if it isn’t alreadyavailable.::   pip install torch    Steps-----1. Import all necessary libraries for loading our data2. Define and intialize the neural network3. Initialize the optimizer4. Save multiple models5. Load multiple models1. Import necessary libraries for loading our data~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~For this recipe, we will use ``torch`` and its subsidiaries ``torch.nn``and ``torch.optim``.<jupyter_code>import torch
import torch.nn as nn
import torch.optim as optim<jupyter_output><empty_output><jupyter_text>2. Define and intialize the neural network~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~For sake of example, we will create a neural network for trainingimages. To learn more see the Defining a Neural Network recipe. Buildtwo variables for the models to eventually save.<jupyter_code>class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.pool = nn.MaxPool2d(2, 2)
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

netA = Net()
netB = Net()<jupyter_output><empty_output><jupyter_text>3. Initialize the optimizer~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~We will use SGD with momentum to build an optimizer for each model wecreated.<jupyter_code>optimizerA = optim.SGD(netA.parameters(), lr=0.001, momentum=0.9)
optimizerB = optim.SGD(netB.parameters(), lr=0.001, momentum=0.9)<jupyter_output><empty_output><jupyter_text>4. Save multiple models~~~~~~~~~~~~~~~~~~~~~~~~~~~Collect all relevant information and build your dictionary.<jupyter_code># Specify a path to save to
PATH = "model.pt"

torch.save({
            'modelA_state_dict': netA.state_dict(),
            'modelB_state_dict': netB.state_dict(),
            'optimizerA_state_dict': optimizerA.state_dict(),
            'optimizerB_state_dict': optimizerB.state_dict(),
            }, PATH)<jupyter_output><empty_output><jupyter_text>4. Load multiple models~~~~~~~~~~~~~~~~~~~~~~~~~~~Remember to first initialize the models and optimizers, then load thedictionary locally.<jupyter_code>modelA = Net()
modelB = Net()
optimModelA = optim.SGD(modelA.parameters(), lr=0.001, momentum=0.9)
optimModelB = optim.SGD(modelB.parameters(), lr=0.001, momentum=0.9)

checkpoint = torch.load(PATH)
modelA.load_state_dict(checkpoint['modelA_state_dict'])
modelB.load_state_dict(checkpoint['modelB_state_dict'])
optimizerA.load_state_dict(checkpoint['optimizerA_state_dict'])
optimizerB.load_state_dict(checkpoint['optimizerB_state_dict'])

modelA.eval()
modelB.eval()
# - or -
modelA.train()
modelB.train()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1616751269035533, 'fraction_numerical': 0.01954314720812183, 'mean_word_length': 4.711594202898551, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '175380', 'license': 'MIT', 'n_tokens_mistral': 1218, 'n_tokens_neox': 1170, 'n_words': 355, 'path': 'PyTorch Recipes/Part 1/saving_multiple_models_in_one_file.ipynb', 'repo_name': 'LbsIrving/PyTorch', 'size': '3940'}