<jupyter_start><jupyter_text>We'll finish by an introduction to PyTorch.  PyTorch and MNISTLet us first build an introductory model for the MNIST dataset.  More on MNIST can be found here: https://www.kaggle.com/hojjatk/mnist-dataset?msclkid=a59b1a61bffd11ec953ecf3f2a143919  PyTorch Tutorials: https://pytorch.org/tutorials/  PyTorch intro: https://www.youtube.com/watch?v=IC0_FRiX-sw - all the Notebooks are available here as well with Deep intros!  Good video with intros: https://www.youtube.com/watch?v=OMDn66kM9Qc   **Loading the MNIST dataset in PyTorch**<jupyter_code>! pip install torch
! pip install torchvision

import torch
from torchvision import transforms
import torch.nn as nn
import torch.nn.functional as f
import torchvision.datasets as datasets<jupyter_output><empty_output><jupyter_text>.ToTensor() - we are using tensors, just as we did in Keras<jupyter_code>batch_size = 128

transform = transforms.ToTensor()

train_data = datasets.MNIST(root='data', train=True,
                                   download=True, transform=transform)
test_data = datasets.MNIST(root='data', train=False,
                                  download=True, transform=transform)
print(train_data)
train_data[0][1]
train_data[0][0].shape<jupyter_output><empty_output><jupyter_text>**DataLoaders in PyTorch** DataLoader - a way to organize data to be easy to use with neural nets<jupyter_code>num_workers = 0

train_loader = torch.utils.data.DataLoader(train_data, batch_size=batch_size,
    num_workers=num_workers, shuffle=True)

test_loader = torch.utils.data.DataLoader(test_data, batch_size=batch_size,
    num_workers=num_workers, shuffle=True)
test_loader.dataset
test_loader.batch_size<jupyter_output><empty_output><jupyter_text>**Building a network**<jupyter_code>class model_mnist(nn.Module):
    def __init__(self):
        super(model_mnist, self).__init__()
        self.input_dense=nn.Linear(28*28,510)
        self.output_dense=nn.Linear(510,10)
        
    def forward(self,x):
        x=f.relu(self.input_dense(x)) 
        pred=self.output_dense(x)
        return pred
model=model_mnist()
print(model.parameters)
learning_rate = 0.01

loss=nn.CrossEntropyLoss()
optimizer=torch.optim.Adam(model.parameters(),lr=learning_rate)
num_epochs = 5
train_loss = list()

for epoch in range(num_epochs):
    total_train_loss = 0
    
    model.train()
    
    # training
    for i, (image, label) in enumerate(train_loader):
        
        image = image.reshape(-1, 28*28)

        # optimizing gradients to zero before training
        optimizer.zero_grad()

        pred = model(image)

        losses = loss(pred, label)
        total_train_loss += losses.item()

        losses.backward()
        optimizer.step()

    total_train_loss = total_train_loss / (i + 1)
    train_loss.append(total_train_loss)

    print (f'Epochs [{epoch+1}/{num_epochs}], Step[{i+1}/{len(train_loader)}], Losses: {losses.item():.6f}')
model.eval()

results = list()
total = 0
for i, (image, label) in enumerate(test_loader):

    image = image.reshape(-1, 28*28)

    pred = model(image)
    pred = torch.nn.functional.softmax(pred, dim=1)

    for j, p in enumerate(pred):
        if label[j] == torch.max(p.data, 0)[1]:
            total = total + 1
            #print(j)
            results.append((image, torch.max(p.data, 0)[1]))

test_accuracy = total / (i*batch_size)
print('Test accuracy {:.4f}'.format(test_accuracy))<jupyter_output>Test accuracy 0.9677
{'fraction_non_alphanumeric': 0.11596298438403702, 'fraction_numerical': 0.021399652978600348, 'mean_word_length': 3.885593220338983, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 4, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '438528', 'license': 'MIT', 'n_tokens_mistral': 1243, 'n_tokens_neox': 1168, 'n_words': 247, 'path': 'Classes/Class-05.19/code/pytorch-intro.ipynb', 'repo_name': 'pamelon/DeepLearning-ALK', 'size': '3458'}