<jupyter_start><jupyter_text>Deep Learning with PyTorchAuthor: [<NAME>](http://teleported.in/)  1.8 Accessing popular datasets via torchvision and torchtext**Objective**: Learn how to access popular datasets via PyTorch utilities **Let's access the CIFAR10 dataset via torchvision**<jupyter_code>import torch
import torchvision
import torchvision.transforms as transforms
import matplotlib.pyplot as plt
# Transforms are applied to each image. Some are mandatory, like conversion to Tensor and 
# Normalize the image. Others are optional and used for data augmentation (like random crop, 
# random jitter etc.)

transform = transforms.Compose([transforms.ToTensor(), 
                                # Various transforms can be added in this pipeline for data augmentation
                                transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])<jupyter_output><empty_output><jupyter_text>`torch.utils.data.Dataset`All datasets (like CIFAR10) are inherited from torch.utils.data.Dataset abstract class. `torch.utils.data.Dataset` represents a dataset, and via the `__len__()` and `__iter__()` functions allows it's client to iterate through the dataset. Custom implementations would read custom datasets and provide a standard interface to it's clients.In the below example, the CIFAR10 class did the work for us in loading CIFAR10 data. It has mechanisms to download the dataset from source, unpack it, and provide us the necessary iterators.<jupyter_code>trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
len(trainset.train_data)
plt.imshow(trainset.train_data[1])
print(trainset.train_labels[1], " - Happens to be the truck class")<jupyter_output>9  - Happens to be the truck class<jupyter_text>`torch.utils.data.DataLoader`DataLoader can load multiple samples parallelly. It provides an iterater to fetch one batch at a time, with the batchsize as specified.DataLoader helps in:* Batching the data* Shuffling the data* Load the data in parallel using multiprocessing workers.<jupyter_code>trainloader = torch.utils.data.DataLoader(trainset, batch_size=10,
                                          shuffle=True, num_workers=2)<jupyter_output><empty_output><jupyter_text>A training loop skeleton<jupyter_code>for i, data in enumerate(trainloader):

    data, labels = data
    
    print("Iteration ", i)
    print("")
    print("type(data): ", type(data))
    print("data.size(): ", data.size())
    print("")
    print("type(labels): ", type(labels))
    print("labels.size(): ", labels.size())
    
    # Model training happens here
    
    break<jupyter_output>Iteration  0

type(data):  <class 'torch.FloatTensor'>
data.size():  torch.Size([10, 3, 32, 32])

type(labels):  <class 'torch.LongTensor'>
labels.size():  torch.Size([10])
{'fraction_non_alphanumeric': 0.10140350877192983, 'fraction_numerical': 0.014035087719298246, 'mean_word_length': 4.100178890876565, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '474644', 'license': 'MIT', 'n_tokens_mistral': 843, 'n_tokens_neox': 804, 'n_words': 265, 'path': '1.8.popular.datasets.ipynb', 'repo_name': 'Masum06/Deep-learning-with-PyTorch-video', 'size': '2854'}