<jupyter_start><jupyter_text>Notas Pytorch (Versión 2)---La mejor forma de implementar redes neuronales en Pytorch manteniendo la flexibilidad y aprovechar las funciones ya implementadas en bajo nivel por el paquete es con `nn.Module`. Existen otras variantes, como implementar los forward pass a mano usando las funciones dentro de `nn.functional` pero con eso hay que mantener el trackeo a mano de los parámetros.<jupyter_code>NUM_TRAIN = 49000 # Cantidad de muestras del training set.

# The torchvision.transforms package provides tools for preprocessing data
# and for performing data augmentation; here we set up a transform to
# preprocess the data by subtracting the mean RGB value and dividing by the
# standard deviation of each RGB value; we've hardcoded the mean and std.
transform = T.Compose([
                T.ToTensor(),
                T.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010))
            ])

# We set up a Dataset object for each split (train / val / test); Datasets load
# training examples one at a time, so we wrap each Dataset in a DataLoader which
# iterates through the Dataset and forms minibatches. We divide the CIFAR-10
# training set into train and val sets by passing a Sampler object to the
# DataLoader telling how it should sample from the underlying Dataset.
cifar10_train = dset.CIFAR10('../assignments/2019/assignment2/cs231n/datasets/', train=True, download=True,
                             transform=transform)
loader_train = DataLoader(cifar10_train, batch_size=64, 
                          sampler=sampler.SubsetRandomSampler(range(NUM_TRAIN)))

cifar10_val = dset.CIFAR10('../assignments/2019/assignment2/cs231n/datasets/', train=True, download=True,
                           transform=transform)
loader_val = DataLoader(cifar10_val, batch_size=64, 
                        sampler=sampler.SubsetRandomSampler(range(NUM_TRAIN, 50000)))

cifar10_test = dset.CIFAR10('../assignments/2019/assignment2/cs231n/datasets/', train=False, download=True, 
                            transform=transform)
loader_test = DataLoader(cifar10_test, batch_size=64)
for (x,y) in loader_train:
    print(x.size())
    print(y.size())
# Uso de GPU:
USE_GPU = True
if USE_GPU and torch.cuda.is_available():
    device = torch.device('cuda')
else:
    device = torch.device('cpu')
    
# Tipo numérico a usar:
dtype = torch.float32 

# Constante para controlar cada cuánto se imprime el valor de la función loss:
print_every = 100<jupyter_output><empty_output><jupyter_text>Ahora explicamos cómo usar el `nn.Module` haciendo una red de dos capas. Ejemplo 1: Red de dos capas1. Primero se define una subclase de `nn.Module`. Para eso se inicializa como subclase en el método `__init__()`.2. Después definimos el forward pass en otro método.<jupyter_code>import torch.nn.functional as F

class TwoLayerFC(nn.Module):
    
    def __init__(self, input_size, hidden_size, num_classes):
        super().__init__()
        # assign layer objects to class attributes
        self.fc1 = nn.Linear(input_size, hidden_size)
        # nn.init package contains convenient initialization methods
        # http://pytorch.org/docs/master/nn.html#torch-nn-init 
        nn.init.kaiming_normal_(self.fc1.weight)
        self.fc2 = nn.Linear(hidden_size, num_classes)
        nn.init.kaiming_normal_(self.fc2.weight)
    
    def forward(self, x):
        # forward always defines connectivity
        x = x.view(x.shape[0],-1)
        scores = self.fc2(F.relu(self.fc1(x)))
        return scores

def test_TwoLayerFC():
    input_size = 50
    x = torch.zeros((64, input_size), dtype=dtype)  # minibatch size 64, feature dimension 50
    model = TwoLayerFC(input_size, 42, 10)
    scores = model(x)
    print(scores.size())  # you should see [64, 10]
    
test_TwoLayerFC()
def check_accuracy_part34(loader, model):
    if loader.dataset.train:
        print('Checking accuracy on validation set')
    else:
        print('Checking accuracy on test set')   
    num_correct = 0
    num_samples = 0
    model.eval()  # set model to evaluation mode
    with torch.no_grad():
        for x, y in loader:
            x = x.to(device=device, dtype=dtype)  # move to device, e.g. GPU
            y = y.to(device=device, dtype=torch.long)
            scores = model(x)
            _, preds = scores.max(1)
            num_correct += (preds == y).sum()
            num_samples += preds.size(0)
        acc = float(num_correct) / num_samples
        print('Got %d / %d correct (%.2f)' % (num_correct, num_samples, 100 * acc))
        
        
def train_part34(model, optimizer, epochs=1):
    """
    Train a model on CIFAR-10 using the PyTorch Module API.
    
    Inputs:
    - model: A PyTorch Module giving the model to train.
    - optimizer: An Optimizer object we will use to train the model
    - epochs: (Optional) A Python integer giving the number of epochs to train for
    
    Returns: Nothing, but prints model accuracies during training.
    """
    model = model.to(device=device)  # move the model parameters to CPU/GPU
    for e in range(epochs):
        for t, (x, y) in enumerate(loader_train):
            model.train()  # put model to training mode
            x = x.to(device=device, dtype=dtype)  # move to device, e.g. GPU
            y = y.to(device=device, dtype=torch.long)

            scores = model(x)
            loss = F.cross_entropy(scores, y)

            # Zero out all of the gradients for the variables which the optimizer
            # will update.
            optimizer.zero_grad()

            # This is the backwards pass: compute the gradient of the loss with
            # respect to each  parameter of the model.
            loss.backward()

            # Actually update the parameters of the model using the gradients
            # computed by the backwards pass.
            optimizer.step()

            if t % print_every == 0:
                print('Iteration %d, loss = %.4f' % (t, loss.item()))
                check_accuracy_part34(loader_val, model)
                print()
                
                
hidden_layer_size = 4000
learning_rate = 1e-2
model = TwoLayerFC(3 * 32 * 32, hidden_layer_size, 10)
optimizer = optim.SGD(model.parameters(), lr=learning_rate)

train_part34(model, optimizer)<jupyter_output>Iteration 0, loss = 3.2691
Checking accuracy on validation set
Got 109 / 1000 correct (10.90)

Iteration 100, loss = 2.1524
Checking accuracy on validation set
Got 303 / 1000 correct (30.30)

Iteration 200, loss = 1.6203
Checking accuracy on validation set
Got 383 / 1000 correct (38.30)

Iteration 300, loss = 1.9937
Checking accuracy on validation set
Got 423 / 1000 correct (42.30)

Iteration 400, loss = 1.8083
Checking accuracy on validation set
Got 418 / 1000 correct (41.80)

Iteration 500, loss = 2.1023
Checking accuracy on validation set
Got 399 / 1000 correct (39.90)

Iteration 600, loss = 1.5131
Checking accuracy on validation set
Got 416 / 1000 correct (41.60)

Iteration 700, loss = 1.4491
Checking accuracy on validation set
Got 495 / 1000 correct (49.50)
{'fraction_non_alphanumeric': 0.08966199971715458, 'fraction_numerical': 0.04341677273370103, 'mean_word_length': 2.8476605005440696, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '253228', 'license': 'MIT', 'n_tokens_mistral': 2404, 'n_tokens_neox': 2142, 'n_words': 717, 'path': 'Utils/Notas-v2-Pytorch.ipynb', 'repo_name': 'LautaroEst/cs231n-Computer-Vision', 'size': '7071'}