<jupyter_start><jupyter_text>Intro in PyTorch, DevOps, R&D in Deep Learning In this workshop, we will introduce pytorch using the classification problem as an example.All files are available on the [class GitHub page.](https://github.com/TahaMHusain/dla)  SetupWe start by building the Python environment. Get `windows_requirements.txt` or `mac_linux_requirements.txt` from the GitHub page - the only difference is the C++ extension that `torchaudio` uses for audio I/O (`soundfile` in Windows and `sox` in Mac/Linux).Once you have the file, run these commands in the terminal or gitbash:```bashconda create -n dla -c conda-forge jupyterlab ipywidgetsconda activate dlapip install -r windows_requirements.txt ` or `ipython kernel install --user --name=dla```If that doesn't work, try building from scratch:```bashconda create -n dla pytorch torchaudio torchvision -c pytorchconda activate dlaconda install -c conda-forge jupyterlab ipywidgets matplotlib scipypip install soundfile ` or `ipython kernel install --user --name=dla```Note:* Windows needs jupyterlab to be installed by conda (Linux doesn't care, haven't tested on Mac)* Whether or not you build from requirements, make sure soundfile/sox is installed with pip* The ipython command adds your new environment to jupyterlab. Make sure to select it in the top right!  CIFAR10 and PyTorch<jupyter_code>from itertools import islice # useful for slicing pytorch DataLoader objects

from matplotlib import pyplot as plt
import numpy as np
import torch
import torchvision
from torchvision import transforms

from torch import nn
from matplotlib import pyplot as plt
import torch.nn.functional as F

import torch.optim as optim<jupyter_output><empty_output><jupyter_text>One of the important components of the neural network training pipeline is data.     There are several stages in pytorch, depending on the task. In this case, we have     image preprocessing in the form of `transform`, reading of images`torchvision.datasets.CIFAR10` and assembly     images in batch `torch.utils.data.DataLoader`.<jupyter_code>transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])

train_dataset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                             download=True, transform=transform)

train_dataloader = torch.utils.data.DataLoader(train_dataset, batch_size=32,
                                               shuffle=True, num_workers=2)

test_dataset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                            download=True, transform=transform)

test_dataloader = torch.utils.data.DataLoader(test_dataset, batch_size=32,
                                              shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse',
           'ship', 'truck')<jupyter_output>Files already downloaded and verified
Files already downloaded and verified<jupyter_text>Let's look at one batch of images<jupyter_code>from itertools import islice
def imshow(img):
    img = img / 2 + 0.5  # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))
    plt.show()
images, _ = list(islice(train_dataloader, 1))[0]

plt.figure(figsize=(10, 10))
imshow(torchvision.utils.make_grid(images))<jupyter_output><empty_output><jupyter_text>The next component is the model itself.<jupyter_code>from torch import nn
import torch.nn.functional as F


class Reshape(nn.Module):
    
    def __init__(self):
        super().__init__()
    
    def forward(self, x):
        # X.shape == [B, C, H, W] -> [B, H']
        return torch.flatten(x, start_dim=1)


class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        
        self.net = nn.Sequential(
            nn.Conv2d(3, 6, 5),
            nn.ReLU(True),
            nn.MaxPool2d(2, 2),
            nn.Conv2d(6, 16, 5),
            nn.ReLU(True),
            nn.MaxPool2d(2, 2),
            Reshape(),
            nn.Linear(16 * 5 * 5, 120),
            nn.ReLU(True),
            nn.Linear(120, 84),
            nn.ReLU(True),
            nn.Linear(84, 10),
        )
        
    def forward(self, x):
        return self.net(x)


net = Net()
device = torch.device('cpu')
net = net.to(device)
net = net.train()<jupyter_output><empty_output><jupyter_text>Determine the loss function and optimization method.<jupyter_code>criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9,
                      nesterov=True, weight_decay=1e-5)<jupyter_output><empty_output><jupyter_text>And the last component of the pipeline is writing train / val loops. In this case, everything is simple, but     in large tasks, this part takes a lot of time and code :( It is worth taking a closer look at the libraries that     they do it for us, for example, pytorch-lighting<jupyter_code>for epoch in range(2):  # loop over the dataset multiple times

    running_loss = 0.0
    for i, data in enumerate(train_dataloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data
        inputs = inputs.to(device)
        labels = labels.to(device)

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        
        optimizer.step()

        # print statistics
        running_loss += loss.item()
        if i % 2000 == 1999:  # print every 2000 mini-batches
            print('[%d, %5d] loss: %.3f' %
                  (epoch + 1, i + 1, running_loss / 2000))
            running_loss = 0.0

print('Finished Training')<jupyter_output>Finished Training<jupyter_text>Let's see the quality of the classification on one test batch<jupyter_code>images, _ = list(islice(test_dataloader, 1))[0]

with torch.no_grad():
    logits = net(images)
    predicted_labels = torch.argmax(logits, dim=1)
_, axes = plt.subplots(4, 8, figsize=(25, 14), dpi=80)
axes = axes.flatten()

for ax, image, label in zip(axes, images, predicted_labels):
    image = image / 2 + 0.5
    image = image.permute(1, 2, 0)
    
    ax.set_title(classes[label.item()])
    ax.imshow(image)
    
plt.show()<jupyter_output><empty_output><jupyter_text>torchaudioLet's get acquainted a little with torchaudio - analogue of torchvision, but for sound. torchaudio provides the ability to read     STFT (short-time Fourier transform), Spectrograms, MelSpectrograms, and much more.<jupyter_code>import torchaudio
from scipy.signal import hann
from scipy.io import wavfile

# torchaudio.set_audio_backend("soundfile")<jupyter_output><empty_output><jupyter_text>We read the wav file and see how it looks<jupyter_code>filename = "audio.wav"
torchaudio.list_audio_backends()
waveform, sample_rate = torchaudio.load(filename)
plt.figure(figsize=(20, 5))
plt.plot(waveform.squeeze())
plt.xlabel('Time')
plt.ylabel('Amplitude')
plt.show()<jupyter_output><empty_output><jupyter_text>Short-time Fourier transformLet's look at the STFT from the box. Let's remember that STFT is cutting audio into windows with some step,     using a window function, and then applying a Fourier transform.<jupyter_code>window = hann(1024)
plt.plot(window)
plt.show()
freqs = torch.stft(
    input=waveform,
    n_fft=1024,
    hop_length=256,
    win_length=1024,
    window=torch.from_numpy(window),
    center=False
)<jupyter_output><empty_output><jupyter_text>To get rid of the complex plane, we take the norm (p == 2) in the last dimension.<jupyter_code>freqs = torch.stft(
    input=waveform,
    n_fft=1024,
    hop_length=256,
    win_length=1024,
    window=torch.from_numpy(window),
    center=False
)
spectrograms = torch.norm(freqs, 2, -1)<jupyter_output><empty_output><jupyter_text>If you look closely, you can see white spots.<jupyter_code>plt.imshow(spectrograms.squeeze())<jupyter_output><empty_output><jupyter_text>Spectrograms<jupyter_code>spectrogramer = torchaudio.transforms.Spectrogram(
    n_fft=1024,
    win_length=1024,
    hop_length=256,
    power=2.0,
)

spectrogram = spectrogramer(waveform)

print("Shape of spectrogram: {}".format(spectrogram.size()))

plt.figure(figsize=(20, 5))
plt.imshow(np.log(spectrogram.squeeze()))
plt.show()<jupyter_output>Shape of spectrogram: torch.Size([1, 513, 833])<jupyter_text>MelSpectrograms<jupyter_code>mel_spectrogramer = torchaudio.transforms.MelSpectrogram(
    sample_rate=22050,
    n_fft=1024,
    win_length=1024,
    hop_length=256,
    f_min=0,
    f_max=8000,
    n_mels=80,
)

mel_spectrogram = mel_spectrogramer(waveform)

print("Shape of spectrogram: {}".format(mel_spectrogram.size()))

plt.figure(figsize=(20, 5))
plt.imshow(torch.log(mel_spectrogram.squeeze()))
plt.show()<jupyter_output>Shape of spectrogram: torch.Size([1, 80, 833])
{'fraction_non_alphanumeric': 0.10749551166965889, 'fraction_numerical': 0.02333931777378815, 'mean_word_length': 3.688584955286691, 'pattern_counts': {'":': 0, '<': 53, '<?xml version=': 0, '>': 54, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 12, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '268072', 'license': 'MIT', 'n_tokens_mistral': 3114, 'n_tokens_neox': 2891, 'n_words': 779, 'path': 'week01/seminar.ipynb', 'repo_name': 'TahaMHusain/dla', 'size': '8912'}