<jupyter_start><jupyter_text>**Feedforward Neural Networks** Importing required libraries<jupyter_code>import os
import torch
from torch import nn
from torch.utils.data import DataLoader
from torchvision import datasets, transforms<jupyter_output><empty_output><jupyter_text>Checking type of device<jupyter_code>device = 'cuda' if torch.cuda.is_available() else 'cpu'
print('Using {} device'.format(device))<jupyter_output>Using cpu device<jupyter_text>Defining nueral network architecture<jupyter_code>class NeuralNetwork(nn.Module):
    """
    the __init__ method defines the network:- function for flattening, network architecture
    
    forward method:- 1st flattens the input then applies the net structure
    """
    
    def __init__(self):
        super(NeuralNetwork, self).__init__()
        self.flatten = nn.Flatten()
        self.linear_relu_stack = nn.Sequential(
            nn.Linear(28*28, 512),
            nn.ReLU(),
            nn.Linear(512, 512),
            nn.ReLU(),
            nn.Linear(512, 10),
            nn.ReLU()
        )

    def forward(self, x):
        x = self.flatten(x)
        logits = self.linear_relu_stack(x)
        return logits<jupyter_output><empty_output><jupyter_text>Creating network instance, move it to device and print the structure<jupyter_code>model = NeuralNetwork().to(device)
print(model)<jupyter_output>NeuralNetwork(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_relu_stack): Sequential(
    (0): Linear(in_features=784, out_features=512, bias=True)
    (1): ReLU()
    (2): Linear(in_features=512, out_features=512, bias=True)
    (3): ReLU()
    (4): Linear(in_features=512, out_features=10, bias=True)
    (5): ReLU()
  )
)<jupyter_text>* Passing input data* Calling the model on the input data ------- returns a 10 dimensional tensor with predicted value to each class* Getting prediction probabilities using `nn.Softmax`<jupyter_code>X = torch.rand(1, 28, 28, device=device)
logits = model(X)
pred_probab = nn.Softmax(dim=1)(logits)
y_pred = pred_probab.argmax(1)
print(f"Predicted class: {y_pred}")<jupyter_output>Predicted class: tensor([0])<jupyter_text>Using 3 images of size 28 x 28.<jupyter_code>input_image = torch.rand(3,28,28)
print(input_image.size())<jupyter_output>torch.Size([3, 28, 28])<jupyter_text>Using `nn.Flatten` to convert input image to 2D image.<jupyter_code>flatten = nn.Flatten()
flat_image = flatten(input_image)
print(flat_image.size())<jupyter_output>torch.Size([3, 784])<jupyter_text>Applying linear transformation on the input<jupyter_code>layer1 = nn.Linear(in_features=28*28, out_features=20)
hidden1 = layer1(flat_image)
print(hidden1.size())<jupyter_output>torch.Size([3, 20])<jupyter_text>Applying non-linear activations ----- creating complex mapping between input and output<jupyter_code>print(f"Before ReLU: {hidden1}\n\n")
hidden1 = nn.ReLU()(hidden1)
print(f"After ReLU: {hidden1}")<jupyter_output>Before ReLU: tensor([[-0.2296, -0.6312, -0.4354, -0.6535, -0.2458,  0.2998,  0.2098, -0.2778,
          0.4642,  0.3735, -0.9103, -0.1168, -0.0728, -0.0995,  0.1878, -0.6349,
          0.0741,  0.0706, -0.1252,  0.0095],
        [-0.0676,  0.0137, -0.4127, -0.6104, -0.1821,  0.1474,  0.3808, -0.5138,
          0.5574,  0.2204, -0.8056, -0.5608,  0.0724, -0.2990, -0.0166, -0.4706,
          0.4690, -0.5249, -0.2011, -0.1336],
        [-0.1517, -0.2456, -0.6936, -0.3848, -0.0984,  0.1003,  0.2210, -0.4613,
          0.4091,  0.3719, -0.8454, -0.4985,  0.0157, -0.0777,  0.5253, -0.5823,
          0.3175, -0.3036, -0.2054, -0.0566]], grad_fn=<AddmmBackward>)


After ReLU: tensor([[0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.2998, 0.2098, 0.0000, 0.4642,
         0.3735, 0.0000, 0.0000, 0.0000, 0.0000, 0.1878, 0.0000, 0.0741, 0.0706,
         0.0000, 0.0095],
        [0.0000, 0.0137, 0.0000, 0.0000, 0.0000, 0.1474, 0.3808, 0.0000, 0.5574,
         0.2204, 0.0000, 0.0000, 0.0724, 0.0000, 0.000[...]<jupyter_text>Applying the network module<jupyter_code>seq_modules = nn.Sequential(
    flatten,
    layer1,
    nn.ReLU(),
    nn.Linear(20, 10)
)
input_image = torch.rand(3,28,28)
logits = seq_modules(input_image)<jupyter_output><empty_output><jupyter_text>* Last linear layer of network* Representing modules predicted probabilities* `dim` parameter indicates the dimension along which the value must sum to 1* The logits are scaled to values [0, 1]<jupyter_code>softmax = nn.Softmax(dim=1)
pred_probab = softmax(logits)<jupyter_output><empty_output><jupyter_text>Weights and biases are optimized during training.<jupyter_code>print("Model structure: ", model, "\n\n")

for name, param in model.named_parameters():
    print(f"Layer: {name} | Size: {param.size()} | Values : {param[:2]} \n")<jupyter_output>Model structure:  NeuralNetwork(
  (flatten): Flatten(start_dim=1, end_dim=-1)
  (linear_relu_stack): Sequential(
    (0): Linear(in_features=784, out_features=512, bias=True)
    (1): ReLU()
    (2): Linear(in_features=512, out_features=512, bias=True)
    (3): ReLU()
    (4): Linear(in_features=512, out_features=10, bias=True)
    (5): ReLU()
  )
) 


Layer: linear_relu_stack.0.weight | Size: torch.Size([512, 784]) | Values : tensor([[-0.0247, -0.0079,  0.0188,  ..., -0.0337, -0.0239,  0.0250],
        [ 0.0237, -0.0318,  0.0121,  ...,  0.0180,  0.0139, -0.0298]],
       grad_fn=<SliceBackward>) 

Layer: linear_relu_stack.0.bias | Size: torch.Size([512]) | Values : tensor([0.0213, 0.0296], grad_fn=<SliceBackward>) 

Layer: linear_relu_stack.2.weight | Size: torch.Size([512, 512]) | Values : tensor([[-0.0163,  0.0125, -0.0325,  ..., -0.0148,  0.0257, -0.0165],
        [ 0.0172, -0.0239,  0.0358,  ..., -0.0279, -0.0263,  0.0055]],
       grad_fn=<SliceBackward>) 

Layer: linear_relu_st[...]
{'fraction_non_alphanumeric': 0.1629203694023349, 'fraction_numerical': 0.13120752744380554, 'mean_word_length': 3.803347280334728, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '300852', 'license': 'Apache-2.0', 'n_tokens_mistral': 2696, 'n_tokens_neox': 2269, 'n_words': 453, 'path': 'pytorch/neural-networks/python/feedforward-neural-network.ipynb', 'repo_name': 'khushi-411/tutorials', 'size': '5739'}