<jupyter_start><jupyter_text>AutoencodersFor feature detection, recommender systems, encoding.Hidden node should not more input nodes, that might overcomplete it. For that we use:* Sparse Autoencoders : Regularize by not using all hidden nodes.* Denoising Autoencoders : Random selection of input nodes* Contractive Autoencoders : adds penalty into the loss function backpropagated* Stacked : Adding more hidden layers 1. We start with an array where the lines correspond to the users and the columns correspond to the movies. Each cell contains ratings from 1 to 5 of movie i by user u.2. The first user goes into the network. The input vector x contains all its ratings for all the movies.3. The input vector x is encoded into a vector z of lower dimensions by mapping function f (e.g. sigmoid function, tanh) z- fW + b where w is weight vector of input weights and b the bias4. z is then decoded into output vector y of same dimension as x, aiming to replicate the input vector x.5. the reconstruction error is d(x,y) = ||x-y|| is computed. The goal is to minimize it.6. The error is back propagated. The weights are updated according to how much they are responsibe for the error. The learning rate decides by how much update the weights.7. The repeat 1 to 6. Reinforcement or Batch Learning.8. When whole dataset is passed through ANN the epoch.<jupyter_code># Importing the libraries
import numpy as np
import pandas as pd
import torch
import torch.nn as nn
import torch.nn.parallel
import torch.optim as optim
import torch.utils.data
from torch.autograd import Variable
# Importing the dataset
movies = pd.read_csv('ml-1m/movies.dat', sep = '::', header = None, engine = 'python', encoding = 'latin-1')
users = pd.read_csv('ml-1m/users.dat', sep = '::', header = None, engine = 'python', encoding = 'latin-1')
ratings = pd.read_csv('ml-1m/ratings.dat', sep = '::', header = None, engine = 'python', encoding = 'latin-1')
movies.head()
users.head()
ratings.head()
# Preparing the training set and the test set
training_set = pd.read_csv('ml-100k/u1.base', delimiter = '\t')
training_set = np.array(training_set, dtype = 'int') #pytorch takes inputs as array
test_set = pd.read_csv('ml-100k/u1.test', delimiter = '\t')
test_set = np.array(test_set, dtype = 'int')
training_set
test_set
# Getting the number of users and movies
nb_users = int(max(max(training_set[:,0]), max(test_set[:,0])))
nb_movies = int(max(max(training_set[:,1]), max(test_set[:,1])))<jupyter_output><empty_output><jupyter_text>List of User 1: [Ratings of all the movies by User 1]List of User 2: [Ratings of all the movies by User 2]................................................................................List of User 943: [Ratings of all the movies by User 943]The new structure of data, having the shape of a 2d array where:the rows are the users,the columns are the movies,the cells are the ratings.<jupyter_code># Converting the data into an array with users in lines and movies in columns
def convert(data):
    new_data = []
    for id_users in range(1, nb_users + 1):
        id_movies = data[:,1][data[:,0] == id_users]
        id_ratings = data[:,2][data[:,0] == id_users]
        ratings = np.zeros(nb_movies)
        ratings[id_movies - 1] = id_ratings
        new_data.append(list(ratings))
    return new_data
training_set = convert(training_set)
test_set = convert(test_set)
training_set[0][0:3]
# Converting the data into Torch tensors
training_set = torch.FloatTensor(training_set)
test_set = torch.FloatTensor(test_set)
training_set
# Creating the architecture of the Neural Network (Sparse Autoencoders)

class SAE(nn.Module):
    def __init__(self, ):
        super(SAE, self).__init__()
        self.fc1 = nn.Linear(nb_movies, 20)
        self.fc2 = nn.Linear(20, 10)
        self.fc3 = nn.Linear(10, 20)
        self.fc4 = nn.Linear(20, nb_movies)
        self.activation = nn.Sigmoid()
    def forward(self, x):
        x = self.activation(self.fc1(x))
        x = self.activation(self.fc2(x))
        x = self.activation(self.fc3(x))
        x = self.fc4(x)
        return x
sae = SAE()
criterion = nn.MSELoss()
optimizer = optim.RMSprop(sae.parameters(), lr = 0.01, weight_decay = 0.5)
# Training the SAE
# With unsqueeze you parse a dimension into your tensor, and therefore you need to set the shape of your indexed tensor at 
# index 0 for both target and input equal, to run tensors with each other.
nb_epoch = 100
for epoch in range(1, nb_epoch + 1):
    train_loss = 0
    s = 0.
    for id_user in range(nb_users):
        input = Variable(training_set[id_user]).unsqueeze(0)
        target = input.clone()
        if torch.sum(target.data > 0) > 0:
            output = sae(input)
            target.require_grad = False
            output[target == 0] = 0
            loss = criterion(output, target)
            mean_corrector = nb_movies/float(torch.sum(target.data > 0) + 1e-10)
            loss.backward()
            train_loss += np.sqrt(loss.item()*mean_corrector)
            s += 1.
            optimizer.step()
    print('epoch: '+str(epoch)+' loss: '+str(train_loss/s))
# Testing the SAE
test_loss = 0
s = 0.
for id_user in range(nb_users):
    input = Variable(training_set[id_user]).unsqueeze(0)
    target = Variable(test_set[id_user]).unsqueeze(0)
    if torch.sum(target.data > 0) > 0:
        output = sae(input)
        target.require_grad = False
        output[target == 0] = 0
        loss = criterion(output, target)
        mean_corrector = nb_movies/float(torch.sum(target.data > 0) + 1e-10)
        test_loss += np.sqrt(loss.item()*mean_corrector)
        s += 1.
print('test loss: '+str(test_loss/s))<jupyter_output>test loss: 0.9604290318022629
{'fraction_non_alphanumeric': 0.11988047108454913, 'fraction_numerical': 0.02179644928809984, 'mean_word_length': 3.390432098765432, 'pattern_counts': {'":': 0, '<': 8, '<?xml version=': 0, '>': 14, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '281393', 'license': 'Apache-2.0', 'n_tokens_mistral': 1877, 'n_tokens_neox': 1791, 'n_words': 617, 'path': 'Pytorch/AutoEncoders/ae.ipynb', 'repo_name': 'SomaKorada07/Deep-Learning-Projects', 'size': '5689'}