<jupyter_start><jupyter_text>Movie Recommendation System ![title](https://movielens.org/images/site/main-screen.png)  Abstract In this notebook, we have analysed the data from movielens dataset and tried to implement recommender system for movies. For this dataset, our main challenge is sparsity of user-movie matrix which means that the number of movie which are rated by user are very less compared to number of movies which are not rated by that same user. As part of data pre-processing, various type of matrix formation has been implemented as per the requirement of all algorithms used in this notebook. we have tried to analyse both neural network and tradition algorithms for recommender system in this notebook. In deep learning, we have implemented Restricted boltzmann machine, Deep Autoencoder, and simple deep neural network to predict the ratings and provide the recommendation based of predicted ratings for the user which has not rated that movie before in the dataset. In traditional algorithms, we have implemented various method of collaborative filtering such as SVD Matrix factorization, SVD++ and Hybrid model. Our analysis is centric towards the results achieved from deep learning algorithms and traditional algorithms by comparing the MSE cost function for all.  Table of Contents1. Acquiring the Data2. Loading the Data3. The Restricted Boltzmann Machine model5.  DNN Model3. Deep Autoencoder4. SVD5. SVD ++6. Hybrid Model3. Conclusion5. Contribution Statement5. Scope3. Citation3. Licence  Acquiring the DataTo start, we need to download the data we are going to use for our system. The datasets we are going to use were acquired by [MovieLens](http://files.grouplens.org/datasets/movielens/) and contain movies, users and movie ratings by these users. With the datasets in place, let's now import the necessary libraries.<jupyter_code>#Tensorflow library. Used to implement machine learning models
import tensorflow as tf
#Numpy contains helpful functions for efficient mathematical calculations
import numpy as np
#Dataframe manipulation library
import pandas as pd
#Graph plotting library
import matplotlib.pyplot as plt
%matplotlib inline

import keras
from IPython.display import SVG
from keras.optimizers import Adam
from keras.utils.vis_utils import model_to_dot

from sklearn.metrics import mean_squared_error as MSE,mean_absolute_error
from tabulate import tabulate
from sklearn.model_selection import train_test_split<jupyter_output><empty_output><jupyter_text>Loading the DataLet's begin by loading in our data with Pandas. The .dat files containing our data are similar to CSV files, but instead of using the ',' (comma) character to separate entries, it uses '::' (two colons) characters instead. To let Pandas know that it should separate data points at every '::', we have to specify the sep='::' parameter when calling the function.Additionally, we also pass it the header=None parameter due to the fact that our files don't contain any headers.Let's start with the movies.dat file and take a look at its structure:<jupyter_code>#Loading in the movies dataset
movies_df = pd.read_csv('../ml-1m/movies.dat', sep='::', header=None, engine='python')
movies_df.head()<jupyter_output><empty_output><jupyter_text>We can do the same for the ratings.dat file:<jupyter_code># Loading the ratings dataset
ratings_df = pd.read_csv('../ml-1m/ratings.dat', sep='::', header=None, engine='python')
ratings_df.head()<jupyter_output><empty_output><jupyter_text>So our movies_df variable contains a dataframe that stores a movie's unique ID number, title and genres, while our ratings_df variable stores a unique User ID number, a movie's ID that the user has watched, the user's rating to said movie and when the user rated that movie.Let's now rename the columns in these dataframes so we can better convey their data more intuitively:<jupyter_code>movies_df.columns = ['MovieID', 'Title', 'Genres']
movies_df.head()<jupyter_output><empty_output><jupyter_text>And our final ratings_df:<jupyter_code>ratings_df.columns = ['UserID', 'MovieID', 'Rating', 'Timestamp']
ratings_df.head()<jupyter_output><empty_output><jupyter_text>The Restricted Boltzmann Machine model![title](https://image.slidesharecdn.com/ywcc5eqte2shgzp7cj0w-signature-d9671bf7a659ac86862f44fb333a532bc81113779840cd708cb67b6aff0ce796-poli-180817162756/95/deep-learning-az-boltzmann-machines-module-5-21-638.jpg?cb=1534525031) The Restricted Boltzmann Machine model has two layers of neurons, one of which is what we call a visible input layer and the other is called a hidden layer. The hidden layer is used to learn features from the information fed through the input layer. For our model, the input is going to contain X neurons, where X is the amount of movies in our dataset. Each of these neurons will possess a normalized rating value varying from 0 to 1, where 0 meaning that a user has not watched that movie and the closer the value is to 1, the more the user likes the movie that neuron's representing. These normalized values, of course, will be extracted and normalized from the ratings dataset.After passing in the input, we train the RBM on it and have the hidden layer learn its features. These features are what we use to reconstruct the input, which in our case, will predict the ratings for movies that user hasn't watched, which is exactly what we can use to recommend movies!We will now begin to format our dataset to follow the model's expected input.  Formatting the DataFirst let's see how many movies we have and see if the movie ID's correspond with that value:<jupyter_code>len(movies_df)<jupyter_output><empty_output><jupyter_text>Now, we can start formatting the data into input for the RBM. We're going to store the normalized users ratings into as a matrix of user-rating called trX, and normalize the values.<jupyter_code>user_rating_df = ratings_df.pivot(index='UserID', columns='MovieID', values='Rating')
user_rating_df.head()
user_rating_df[user_rating_df.index==215]<jupyter_output><empty_output><jupyter_text>Lets normalize it now:<jupyter_code>norm_user_rating_df = user_rating_df.fillna(0) / 5.0
trX = norm_user_rating_df.values
trX[0:5]
trX[215]
#ratings_df[( ratings_df['UserID']==21)].sort_values(by=['MovieID'],ascending=True)<jupyter_output><empty_output><jupyter_text>Setting the Model's ParametersNext, let's start building our RBM with TensorFlow. We'll begin by first determining the number of neurons in the hidden layers and then creating placeholder variables for storing our visible layer biases, hidden layer biases and weights that connects the hidden layer with the visible layer. We will be arbitrarily setting the number of neurons in the hidden layers to 20. You can freely set this value to any number you want since each neuron in the hidden layer will end up learning a feature.<jupyter_code># Number of hidden nodes
hiddenNodes = 20
# Number of Visible nodes
visibleNodes =  len(user_rating_df.columns)
#Number of unique movies
visibleBias = tf.placeholder("float", [visibleNodes])
#Number of features we're going to learn
hiddenBias = tf.placeholder("float", [hiddenNodes])
# Weight Matrix
W = tf.placeholder("float", [visibleNodes, hiddenNodes])<jupyter_output><empty_output><jupyter_text>We then move on to creating the visible and hidden layer units and setting their activation functions. In this case, we will be using the tf.sigmoid and tf.relu functions as nonlinear activations since it is commonly used in RBM's.<jupyter_code>#Phase 1: Input Processing
visible_0 = tf.placeholder("float", [None, visibleNodes])
_hidden_0 = tf.nn.sigmoid(tf.matmul(visible_0, W) + hiddenBias)
hidden_0 = tf.nn.relu(tf.sign(_hidden_0 - tf.random_uniform(tf.shape(_hidden_0))))
#Phase 2: Reconstruction
_v1 = tf.nn.sigmoid(tf.matmul(hidden_0, tf.transpose(W)) + visibleBias) 
visible_1 = tf.nn.relu(tf.sign(_v1 - tf.random_uniform(tf.shape(_v1))))
hidden_1 = tf.nn.sigmoid(tf.matmul(visible_1, W) + hiddenBias)<jupyter_output><empty_output><jupyter_text>Now we set the RBM training parameters and functions.<jupyter_code>#Learning rate
alpha = 1.0
#Create the gradients
w_pos_grad = tf.matmul(tf.transpose(visible_0), hidden_0)
w_neg_grad = tf.matmul(tf.transpose(visible_1), hidden_1)
#Calculate the Contrastive Divergence to maximize
CD = (w_pos_grad - w_neg_grad) / tf.to_float(tf.shape(visible_0)[0])
#Create methods to update the weights and biases
update_weight = W + alpha * CD
update_vb = visibleBias + alpha * tf.reduce_mean(visible_0 - visible_1, 0)
update_hb = hiddenBias + alpha * tf.reduce_mean(hidden_0 - hidden_1, 0)<jupyter_output>WARNING:tensorflow:From <ipython-input-11-939b7ceaabc2>:7: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.<jupyter_text>And set the error function, which in this case will be the Mean Absolute Error Function.<jupyter_code>err = visible_0 - visible_1
err_sum = tf.reduce_mean(err * err)<jupyter_output><empty_output><jupyter_text>We also have to initialize our variables. Thankfully, NumPy has a handy ,code>zeros function for this. We use it like so:<jupyter_code>#Current weight
current_weight = np.zeros([visibleNodes, hiddenNodes], np.float32)
#Current visible unit biases
current_vb = np.zeros([visibleNodes], np.float32)
#Current hidden unit biases
current_hb = np.zeros([hiddenNodes], np.float32)
#Previous weight
previous_weight = np.zeros([visibleNodes, hiddenNodes], np.float32)
#Previous visible unit biases
previous_vb = np.zeros([visibleNodes], np.float32)
#Previous hidden unit biases
previous_hb = np.zeros([hiddenNodes], np.float32)
sess = tf.Session()
sess.run(tf.global_variables_initializer())<jupyter_output><empty_output><jupyter_text>Now we train the RBM with 15 epochs with each epoch using 10 batches with size 100. After training, we print out a graph with the error by epoch.<jupyter_code>epochs = 15
batchsize = 100
errors = []
for i in range(epochs):
    for start, end in zip( range(0, len(trX), batchsize), range(batchsize, len(trX), batchsize)):
        batch = trX[start:end]
        current_weight = sess.run(update_weight, feed_dict={visible_0: batch, W: previous_weight, visibleBias: previous_vb, hiddenBias: previous_hb})
        current_vb = sess.run(update_vb, feed_dict={visible_0: batch, W: previous_weight, visibleBias: previous_vb, hiddenBias: previous_hb})
        cur_nb = sess.run(update_hb, feed_dict={visible_0: batch, W: previous_weight, visibleBias: previous_vb, hiddenBias: previous_hb})
        previous_weight = current_weight
        previous_vb = current_vb
        previous_hb = current_hb
    errors.append(sess.run(err_sum, feed_dict={visible_0: trX, W: current_weight, visibleBias: current_vb, hiddenBias: current_hb}))
    print (errors[-1])
plt.plot(errors)
plt.ylabel('Error')
plt.xlabel('Epoch')
plt.show()<jupyter_output>0.059456352
0.05070744
0.04837974
0.047687545
0.046928756
0.04621484
0.046074197
0.04588732
0.04555632
0.045446653
0.045222387
0.045326088
0.045047265
0.04511021
0.044937387<jupyter_text>RecommendationWe can now predict movies that an arbitrarily selected user might like. This can be accomplished by feeding in the user's watched movie preferences into the RBM and then reconstructing the input. The values that the RBM gives us will attempt to estimate the user's preferences for movies that he hasn't watched based on the preferences of the users that the RBM was trained on.Lets first select a User ID of our mock user:<jupyter_code>mock_user_id = 215
#Selecting the input user
inputUser = trX[mock_user_id-1].reshape(1, -1)
inputUser[0:5]
#Feeding in the user and reconstructing the input
hidden_0 = tf.nn.sigmoid(tf.matmul(visible_0, W) + hiddenBias)
vv1 = tf.nn.sigmoid(tf.matmul(hidden_0, tf.transpose(W)) + visibleBias)
feed = sess.run(hidden_0, feed_dict={ visible_0: inputUser, W: previous_weight, hiddenBias: previous_hb})
rec = sess.run(vv1, feed_dict={ hidden_0: feed, W: previous_weight, visibleBias: previous_vb})
print(rec)<jupyter_output>[[0.5824928  0.24802381 0.07080343 ... 0.00350019 0.00244745 0.01251542]]<jupyter_text>We can then list the 20 most recommended movies for our mock user by sorting it by their scores given by our model.<jupyter_code>scored_movies_df_mock = movies_df[movies_df['MovieID'].isin(user_rating_df.columns)]
scored_movies_df_mock = scored_movies_df_mock.assign(RecommendationScore = rec[0])
scored_movies_df_mock.sort_values(["RecommendationScore"], ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>So, how to recommend the movies that the user has not watched yet?Now, we can find all the movies that our mock user has watched before:<jupyter_code>movies_df_mock = ratings_df[ratings_df['UserID'] == mock_user_id]
movies_df_mock.head()<jupyter_output><empty_output><jupyter_text>In the next cell, we merge all the movies that our mock users has watched with the predicted scores based on his historical data:<jupyter_code>#Merging movies_df with ratings_df by MovieID
merged_df_mock = scored_movies_df_mock.merge(movies_df_mock, on='MovieID', how='outer')<jupyter_output><empty_output><jupyter_text>Lets sort it and take a look at the first 20 rows:<jupyter_code>merged_df_mock.sort_values(["RecommendationScore"], ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>As you can see, there are some movies that user has not watched yet and has high score based on our model. So, we can recommend them to the user.  Simple Deep Neural Network ModelSimple Deep neural network model combining a collaborative filtering recommendation algorithm with deep learning technology, therein consisting of two parts. First, the model uses a feature representation method based on a quadric polynomial regression model,which obtains the latent features more accurately by improving upon the traditional matrix factorization algorithm. Then, these latent features are regarded as the input data of the deep neural network model, which is the second part of the proposed model and is used to predict the rating scores. ![image.png](attachment:image.png)<jupyter_code>len(ratings_df.UserID.unique()), len(ratings_df.MovieID.unique())<jupyter_output><empty_output><jupyter_text>We assign a unique number between (0, users) to each user and do the same for movies and create dataframe which have userid and movieid along with newly generated userid and movieid.<jupyter_code>ratings_df['UserID_C'] = ratings_df.UserID.astype('category').cat.codes.values
ratings_df['MovieID_C'] = ratings_df.MovieID.astype('category').cat.codes.values
user_movie_cate_df = ratings_df[['UserID','UserID_C','MovieID','MovieID_C']]
ratings_df.drop(['UserID','MovieID'],axis=1,inplace=True)
ratings_df.head()<jupyter_output><empty_output><jupyter_text>Train test splitWe'll now split our dataset of ratings into train and test. Given the train set, we'd like to accurately estimate the ratings in the test set.<jupyter_code>train, test = train_test_split(ratings_df, test_size=0.2,random_state=7856)

# True rating for test dataframe
y_true = test.Rating
n_users, n_movies = len(ratings_df.UserID_C.unique()), len(ratings_df.MovieID_C.unique())
train.head()<jupyter_output><empty_output><jupyter_text>Neural networks for recommendationWe'll now create a simple neural network for recommendation, or for estimating rating! This model is very similar to the matrix factorisation models, but differs in the following ways:1. Instead of taking a dot product of the user and the item embedding, we concatenate them and use them as features for our neural network. Thus, we are not constrained to the dot product way of combining the embeddings, and can learn complex non-linear relationships.2. Due to 1, we can now have a different dimension of user and movie embeddings. This can be useful if one dimension is larger than the other.<jupyter_code>n_latent_factors_user = 6
n_latent_factors_movie = 10

movie_input = keras.layers.Input(shape=[1],name='Item')
movie_embedding = keras.layers.Embedding(n_movies + 1, n_latent_factors_movie, name='Movie-Embedding')(movie_input)
movie_vec = keras.layers.Flatten(name='FlattenMovies')(movie_embedding)
movie_vec = keras.layers.Dropout(0.2)(movie_vec)


user_input = keras.layers.Input(shape=[1],name='User')
user_vec = keras.layers.Flatten(name='FlattenUsers')(keras.layers.Embedding(n_users + 1, n_latent_factors_user,name='User-Embedding')(user_input))
user_vec = keras.layers.Dropout(0.2)(user_vec)


concat = keras.layers.concatenate([movie_vec, user_vec])
concat_dropout = keras.layers.Dropout(0.2)(concat)
dense = keras.layers.Dense(200,name='FullyConnected')(concat)
dropout_1 = keras.layers.Dropout(0.2,name='Dropout')(dense)
dense_2 = keras.layers.Dense(100,name='FullyConnected-1')(concat)
dropout_2 = keras.layers.Dropout(0.2,name='Dropout')(dense_2)
dense_3 = keras.layers.Dense(50,name='FullyConnected-2')(dense_2)
dropout_3 = keras.layers.Dropout(0.2,name='Dropout')(dense_3)
dense_4 = keras.layers.Dense(20,name='FullyConnected-3', activation='relu')(dense_3)


result = keras.layers.Dense(1, activation='relu',name='Activation')(dense_4)
adam = Adam(lr=0.005)
model = keras.Model([user_input, movie_input], result)
model.compile(optimizer=adam,loss= 'mean_absolute_error')
model.summary()<jupyter_output>__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
Item (InputLayer)               (None, 1)            0                                            
__________________________________________________________________________________________________
User (InputLayer)               (None, 1)            0                                            
__________________________________________________________________________________________________
Movie-Embedding (Embedding)     (None, 1, 10)        37070       Item[0][0]                       
__________________________________________________________________________________________________
User-Embedding (Embedding)      (None, 1, 6)         36246       User[0][0]                       
__________[...]<jupyter_text>We can see that the number of parameters is more than what we had in the Matrix Factorisation case. Let's see how this model works. I'll run it for more epochs given that we have more parameters.<jupyter_code>history = model.fit([train.UserID_C, train.MovieID_C], train.Rating, epochs=30, verbose=0)<jupyter_output><empty_output><jupyter_text>Prediction performance of Neural Network based recommender system<jupyter_code>y_hat_2 = np.round(model.predict([test.UserID_C, test.MovieID_C]),0)
print(mean_absolute_error(y_true, y_hat_2))

print(mean_absolute_error(y_true, model.predict([test.UserID_C, test.MovieID_C])))
test_user_215= pd.DataFrame({'UserID_C' : [214] * 3706,'MovieID_C':list(ratings_df['MovieID_C'].unique())})
#test_user_215.head()
test_user_215['Predicted_Ratings'] = np.round(model.predict([test_user_215.UserID_C, test_user_215.MovieID_C]),0)
test_user_215.head()
result = pd.merge(test_user_215,test,how='left',on=['UserID_C','MovieID_C'])
result = result[pd.isnull(result['Rating'])]
result.sort_values(by='Predicted_Ratings',ascending=False,inplace=True)
result.head(10)
result.rename(columns={'MovieID_C':'MovieID'},inplace=True)
final_df = pd.merge(result,movies_df,how='inner',on=['MovieID'])
final_df.sort_values(by=['Predicted_Ratings'],ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>Deep Autoencoder![title](https://cdn-images-1.medium.com/max/1000/1*Uz3sGf9XyLyD5AfcUE9n3w.png) An Autoencoder is a profound learning neural system design that accomplishes best in class execution in the territory of collaborative filtering and furthermore used to get familiar with a representation (encoding) for a lot of input data, usually to a achieve dimensionality reduction. Architecturally, the type of an Autoencoder is a feedforward neural system having an input layer, one hidden layer and a output layer.The output layer has the same number of neurons as the input layer for the purpose of reconstructing it’s own inputs. It is useful that an Autoencoder has a smaller hidden layer than the input layer. It is useful that an Autoencoder has a smaller hidden layer than the input layer. This effect forces the model to create a compressed representation of the data in the hidden layer by learning correlations in the data.The core purpose of the algorithm is to reconstruct the original input data.During the training time the encoder takes a input data sample x and maps it to the so called hidden or latent representation z. Then the decoder maps z to the output vector x’ which is (in the best case scenario) the exact representation of the input data x. Please notice that usually an exact recreation of the input x is not possible.The additional hidden layers enable the Autoencoder to learn mathematically more complex underlying patterns in the data.Deeper layers of the Deep Autoencoder tend to learn even higher-order features. Autoencode takes inputs as user-item matrix where below code convert pivot table function to transform the ratings dataframe to user-movie matrix where value in the matrix is of ratings for user-movie pair.<jupyter_code># Lets pivot the data to get it at a user level
ratings_transform = pd.pivot_table(ratings_df[['UserID','MovieID','Rating']],values=['Rating'], index=['UserID'], columns=['MovieID'] ).fillna(0)

# creating train and test sets
X_train_enc, X_test_enc = train_test_split(ratings_transform, train_size=0.8,random_state=7856)
X_train_enc.head()<jupyter_output><empty_output><jupyter_text>Below code initialize the weights and number of neurons for each layer in the autoencoder. we have generated weights for each layer using tf.random_normal which generates tensor of the specified shape with random normal values.<jupyter_code># Nodes for encoding layer
n_nodes_inpl = 3706  
n_nodes_hl1  = 1853  
n_nodes_hl2  = 925

# Nodes for hiddern layer
n_nodes_hl3  = 252  

# Nodes for decoding layer
n_nodes_hl4  = 925
n_nodes_hl5  = 1853
n_nodes_outl = 3706  

# input layer has 9724*4862 weights and 4862 biases
hidden_1_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_inpl,n_nodes_hl1]))}

# second encode layer has 4862*2431 weights and 2431 biases
hidden_2_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_hl1,n_nodes_hl2]))}

# Third encode layer has 2431*512 weights and 512 biases
hidden_3_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_hl2,n_nodes_hl3]))}

# First decode layer has 512*2431 weights and 2431 biases
hidden_4_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_hl3,n_nodes_hl4]))}

# Second decode layer has 2431*4862 weights and 4862 biases
hidden_5_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_hl4,n_nodes_hl5]))}

# output layer has 4862*9724 weights and 9724 biases
output_layer_vals = {'weights':tf.Variable(tf.random_normal([n_nodes_hl5,n_nodes_outl])) }<jupyter_output><empty_output><jupyter_text>Built deep autoencode input, hidden and output layers using below code where one constant node has been created for each layer which represents bias for each layer having value equal to 1. Add bias to all neurons after multiplying the input layer and random normal weight and then apply activation function to add non linerity. This network has MSE a cost function and optimizing it using Adagrad optimizer.<jupyter_code># user with 3706 ratings goes in
input_layer = tf.placeholder('float', [None, 3706])

### First Layer
# add a constant node to the first layer
# it needs to have the same shape as the input layer for me to be
# able to concatinate it later
input_layer_const = tf.fill([tf.shape(input_layer)[0], 1] ,1.0 )
# multiply output of input_layer wth a weight matrix 
layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(input_layer,hidden_1_layer_vals['weights']),input_layer_const))

### Second Layer
input_layer_const1 = tf.fill( [tf.shape(layer_1)[0], 1] ,1.0  )
# multiply output of input_layer wth a weight matrix 
layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1,hidden_2_layer_vals['weights']),input_layer_const1))

### Third Layer
input_layer_const2 = tf.fill( [tf.shape(layer_2)[0], 1] ,1.0  )
# multiply output of input_layer wth a weight matrix 
layer_3 = tf.nn.sigmoid(tf.add(tf.matmul(layer_2,hidden_3_layer_vals['weights']),input_layer_const2))

### Fourth Layer
input_layer_const3 = tf.fill( [tf.shape(layer_3)[0], 1] ,1.0  )
# multiply output of input_layer wth a weight matrix 
layer_4 = tf.nn.sigmoid(tf.add(tf.matmul(layer_3,hidden_4_layer_vals['weights']),input_layer_const3))

### Fifth Layer
input_layer_const4 = tf.fill( [tf.shape(layer_4)[0], 1] ,1.0  )
# multiply output of input_layer wth a weight matrix 
layer_5 = tf.nn.sigmoid(tf.add(tf.matmul(layer_4,hidden_5_layer_vals['weights']),input_layer_const4))

### Output Layer
# adding one bias node to the hidden layer
layer1_const = tf.fill( [tf.shape(layer_5)[0], 1] ,1.0  )s

# multiply output of hidden with a weight matrix to get final output
output_layer = tf.matmul(layer_5,output_layer_vals['weights'])

# output_true shall have the original shape for error calculations
output_true = tf.placeholder('float', [None, 3706])
# define our cost function
meansq =    tf.reduce_mean(tf.square(output_layer - output_true))
# define our optimizer
learn_rate = 0.001   # how fast the model should learn
optimizer = tf.train.AdagradOptimizer(learn_rate).minimize(meansq)
# initialising variables and starting the session
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init)
# defining batch size, number of epochs and learning rate
batch_size = 100  # how many users to use together for training
hm_epochs =100    # how many times to go through the entire dataset
tot_users = X_train_enc.shape[0] # total number of users<jupyter_output><empty_output><jupyter_text>Train and test Deep Autoencoder with 100 epoch value and feed train and test data in order to get your reconstructed movie rating for that user. Also, calculate MSE for training and testing phase.<jupyter_code># running the model for a 100 epochs taking 100 users in batches
# total improvement is printed out after each epoch
for epoch in range(hm_epochs):
    epoch_loss = 0    # initializing error as 0
    
    for i in range(int(tot_users/batch_size)):
        epoch_x = X_train_enc[ i*batch_size : (i+1)*batch_size ]
        _, c = sess.run([optimizer, meansq],feed_dict={input_layer: epoch_x,output_true: epoch_x})
        epoch_loss += c
        
    output_train = sess.run(output_layer,feed_dict={input_layer:X_train_enc})
    output_test = sess.run(output_layer,feed_dict={input_layer:X_test_enc})
        
    print('MSE train', MSE(output_train, X_train_enc),'MSE test', MSE(output_test, X_test_enc))      
    print('Epoch', epoch, '/', hm_epochs, 'loss:',epoch_loss)<jupyter_output>MSE train 717.8827531835316 MSE test 721.2971359886285
Epoch 0 / 100 loss: 39416.19738769531
MSE train 588.8045268721269 MSE test 592.9499724487845
Epoch 1 / 100 loss: 31350.027465820312
MSE train 499.1049062019719 MSE test 503.1666525128692
Epoch 2 / 100 loss: 26144.38507080078
MSE train 433.0062877090002 MSE test 437.11795716411353
Epoch 3 / 100 loss: 22409.0263671875
MSE train 382.2285363298209 MSE test 386.1151389161219
Epoch 4 / 100 loss: 19602.64239501953
MSE train 341.6850645136781 MSE test 345.16777591125276
Epoch 5 / 100 loss: 17407.228973388672
MSE train 308.386510255548 MSE test 311.55543854156986
Epoch 6 / 100 loss: 15630.277893066406
MSE train 280.5592578599356 MSE test 283.42160827273955
Epoch 7 / 100 loss: 14158.085723876953
MSE train 256.87845138032753 MSE test 259.55103518375944
Epoch 8 / 100 loss: 12918.815460205078
MSE train 236.33650461526895 MSE test 238.8395876776703
Epoch 9 / 100 loss: 11854.868789672852
MSE train 218.2814149517687 MSE test 220.6170982261209
Epoc[...]<jupyter_text>Select user with id 215 and get the reconstructed user ratings to compare the RBM model with deep autoencode and compare the recomendation for this user thorough both models. Display reconstructed movie ratings for 215 userid in reversed order to show highest ratings for this userid.<jupyter_code>sample_user = X_test_enc.iloc[774,:]
sample_user_pred = sess.run(output_layer, feed_dict={input_layer:[sample_user]})
temp_list = sample_user_pred[0].tolist()
temp_list.sort(reverse=True)
print(temp_list)
mock_user_id = 215


#Selecting the input user
inputUser = np.asarray(temp_list).reshape(1,-1)
inputUser[0:5]


#Feeding in the user and reconstructing the input
hidden_0 = tf.nn.sigmoid(tf.matmul(visible_0, W) + hiddenBias)
vv1 = tf.nn.sigmoid(tf.matmul(hidden_0, tf.transpose(W)) + visibleBias)
feed = sess.run(hidden_0, feed_dict={ visible_0: inputUser, W: previous_weight, hiddenBias: previous_hb})
rec = sess.run(vv1, feed_dict={ hidden_0: feed, W: previous_weight, visibleBias: previous_vb})
print(rec)


scored_movies_df_mock = movies_df[movies_df['MovieID'].isin(user_rating_df.columns)]
scored_movies_df_mock = scored_movies_df_mock.assign(RecommendationScore = rec[0])
scored_movies_df_mock.sort_values(["RecommendationScore"], ascending=False).head(20)
movies_df_mock = ratings_df[ratings_df['UserID'] == mock_user_id]
movies_df_mock.head()

#Merging movies_df with ratings_df by MovieID
merged_df_mock = scored_movies_df_mock.merge(movies_df_mock, on='MovieID', how='outer')

merged_df_mock.sort_values(["RecommendationScore"], ascending=False).head(20)<jupyter_output><empty_output><jupyter_text>Model-Based Collberative Filtering Latent Semantic ModelModel-based Collaborative Filtering is based on matrix factorization (MF) which has received greater exposure, mainly as an unsupervised learning method for latent variable decomposition and dimensionality reduction. Matrix factorization is widely used for recommender systems where it can deal better with scalability and sparsity than Memory-based CF: - The goal of MF is to learn the latent preferences of users and the latent attributes of items from known ratings (learn features that describe the characteristics of ratings) to then predict the unknown ratings through the dot product of the latent features of users and items.- When you have a very sparse matrix, with a lot of dimensions, by doing matrix factorization, you can restructure the user-item matrix into low-rank structure, and you can represent the matrix by the multiplication of two low-rank matrices, where the rows contain the latent vector.- You fit this matrix to approximate your original matrix, as closely as possible, by multiplying the low-rank matrices together, which fills in the entries missing in the original matrix.  Singular Value Decomposition (SVD)One of the well-known matrix factorization methods is Singular value decomposition (SVD). At a high level, SVD is an algorithm that decomposes a matrix $A$ into the best lower rank (i.e. smaller/simpler) approximation of the original matrix $A$. Mathematically, it decomposes A into a two unitary matrices and a diagonal matrix: Singular value decomposition is a method of decomposing a matrix into three other matrices:![image.png](attachment:image.png)Where:- A is an m × n matrix- U is an m × r orthogonal matrix- S is an r × r diagonal matrix- V is an r × n orthogonal matrix ![image.png](attachment:image.png)<jupyter_code>## Import Libraries
import os

from math import sqrt
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import linear_kernel
from sklearn.metrics import pairwise_distances
from scipy.spatial.distance import cosine, correlation
from scipy.sparse.linalg import svds

from surprise import Reader, Dataset, SVD, evaluate
from surprise.model_selection import cross_validate

from __future__ import (absolute_import, division, print_function,unicode_literals)

from surprise import SVDpp
from surprise import accuracy
from surprise.model_selection import train_test_split
from surprise.model_selection import GridSearchCV
## load data set
movies_df = pd.read_csv('../ml-1m/movies.dat', sep='::',names = ["movieId","title","genres"],header=None, engine='python')
users_df = pd.read_csv('../ml-1m/users.dat', sep='::',names = ["userId","gender","age","occupation","zip-code"],header=None, engine='python')
ratings_df = pd.read_csv('../ml-1m/ratings.dat', sep='::',names=["userId","movieId","rating","timestamp"],header=None, engine='python')
print("movies:",movies_df.shape)
print("users:",users_df.shape)
print("ratingss:",ratings_df.shape)<jupyter_output>movies: (3883, 3)
users: (6040, 5)
ratingss: (1000209, 4)<jupyter_text>Function to define moviesId those have not yet been predicted by the user<jupyter_code>def User_Movie_notpred(UserId,df,rows):
    panda_df = pd.DataFrame(df)
    MoviesforUser = ((panda_df.loc[panda_df['userId'] == UserId])[['movieId']]).values.tolist()
    NotRecMovielist = ((panda_df[~panda_df['movieId'].isin(MoviesforUser)])[['movieId']])['movieId'].unique().tolist()
    return NotRecMovielist[:rows]<jupyter_output><empty_output><jupyter_text>here randomly we have taken userId 215 for predicting the ratings for the movies those have not yet been rated by the user.We restricted the movies Id to 50 rows for convenience.<jupyter_code>User_Movie_notpred(215,ratings_df,15)<jupyter_output><empty_output><jupyter_text>We then set up various lookup tables that we will need to translate ratings into matrix elements and recommendations from the matrix factorization back to human readable form.<jupyter_code># Creating array to store movie id and title
movies_id2name, movies_name2id = {}, {}
ids = movies_df["movieId"].values
titles = movies_df["title"].values
for id, title in zip(ids, titles):
    movies_id2name[id] = title
    movies_name2id[title] = id
    
# Creating index for movieId
movies_id2idx, movies_idx2id = {}, {}
for idx, movie_id in enumerate(ratings_df["movieId"].unique()):
    movies_id2idx[movie_id] = idx
    movies_idx2id[idx] = movie_id
    
# Creating index for userId
users_id2idx, users_idx2id = {}, {}
for idx, user_id in enumerate(ratings_df["userId"].unique()):
    users_id2idx[user_id] = idx
    users_idx2id[idx] = user_id
    
# Calculate the lenth of the users and movies
num_users = len(users_id2idx)
num_movies = len(movies_id2idx)<jupyter_output><empty_output><jupyter_text>Finally, we are ready to construct the matrix. We choose to use a dense matrix full of zeros to start with, where rows represent users and columns represent movies. We have 6040 users and 3706 items, so our matrix R_val is of shape (6040, 3706). We use the lookup tables we just generated to fill out the ratings for each (user, movie) pair that we have information for.<jupyter_code>def construct_original_matrix(num_users, num_movies, ratings_df,users_id2idx, movies_id2idx):
    X = np.zeros((num_users, num_movies), dtype=np.float32)
    for user_id, movie_id, rating in ratings_df[["userId", "movieId", "rating"]].values:
        X[users_id2idx[user_id], movies_id2idx[movie_id]] = rating
    return X
R_val = construct_original_matrix(num_users, num_movies, ratings_df,users_id2idx, movies_id2idx)<jupyter_output><empty_output><jupyter_text>The next step is to compute different kinds of bias and remove it from the matrix. This allows SVD to be more effective, as we will see in the results later. Bias can be split up into global bg, user bu and user-item bui, which represent the bias across all users, bias per user and bias per item per user.                                           b = bg + bu +bui                                                    Where- b  = bias- bg = global bias- bu = user bias- bui = bias per user per item Since this is a sparse matrix, we will treat all zero entries as unknown, and only consider non-zero entries for computing the bias. This is done using the code below. As you can see, this is essentially computing and removing average values of all non-zero entries along different axes.<jupyter_code>#Compute the bias 
def compute_bias(X, bias_type):
    Xc = X.copy()
    Xc[Xc == 0] = np.nan
    if bias_type == "global":
        return np.nanmean(Xc)
    elif bias_type == "user":
        return np.mean(np.nanmean(Xc, axis=0))
    elif bias_type == "item":
        return np.mean(np.nanmean(Xc, axis=1))
    else:
        raise Exception("invalid bias type")
#Remove the bias
def remove_bias(X, bias):
    Xc = X.copy()
    Xc[Xc == 0] = np.nan
    Xc = np.subtract(Xc, bias)
    Xc = np.nan_to_num(Xc)
    return Xc
#Compute and Remove the global bias across the users
bg = compute_bias(R_val, "global")
Rg_val = remove_bias(R_val, bg)

#Compute and remove the user bias
bu = compute_bias(Rg_val, "user")
Ru_val = remove_bias(Rg_val, bu)

#Compute and remove the item bias
bi = compute_bias(Rg_val, "item")
Ri_val = remove_bias(Rg_val, bi)

#Compute and remove the per item per user bias
bui = compute_bias(Ru_val, "item")
Rui_val = remove_bias(Ru_val, bui)<jupyter_output><empty_output><jupyter_text>Now that we have our different debiased input matrices, the next step is to decompose it into its constituents using SVD, and recompose it to an approximation using smaller dimensions. This will give us a matrix whose reduced columns correspond to user-tastes rather than actual movies, so we can make predictions of how a user will rank the movies they haven't seen already, or rank movies according to how the user would like them. We use TF's built-in svd() call here. The graph will factorize the input matrix R using SVD into matrices U, S and V, recompose reduced versions of the three matrices to get an approximation R'. It then computes the reconstruction error (MAE) between R and R', and also the proportion of explained variance R2 and returns it.<jupyter_code>def reconstruct_using_svd(X, k):
    
    if k == 0:
        return X, 1., 0.
    
    graph = tf.Graph()
    with graph.as_default():
        
        # input arg
        R = tf.placeholder(tf.float32, shape=(num_users, num_movies), name="R")
        
        # run SVD
        S, U, Vt = tf.svd(R, full_matrices=True)
        
        # reduce dimensions
        Sk = tf.diag(S)[0:k, 0:k]
        Uk = U[:, 0:k]
        Vk = tf.transpose(Vt)[0:k, :]

        # reconstruct matrix
        Rprime = tf.matmul(Uk, tf.matmul(Sk, Vk))

        
        # compute reconstruction MAE
        rsquared = tf.linalg.norm(Rprime) / tf.linalg.norm(R)
        mae = tf.metrics.mean_absolute_error(R, Rprime)[1]
        
    with tf.Session(graph=graph) as sess:
        
        tf.global_variables_initializer().run()
        tf.local_variables_initializer().run()
        
        [Rprime_val, rsquared_val, mae_val] = sess.run(
            [Rprime, rsquared, mae], feed_dict={R: X})
        return Rprime_val, rsquared_val, mae_val<jupyter_output><empty_output><jupyter_text>Here we are calculating the reconstructed matrix with bias by using the above defined fuction and taking latent feature k = 50.<jupyter_code>R_rec, rsquared, rec_err = reconstruct_using_svd(R_val, 50)
print("reconstruction error (MAE):", rec_err)
print("percentage of variance explained: {:.3f}".format(rsquared))
print("shape of reconstructed matrix: ", R_rec.shape)<jupyter_output>reconstruction error (MAE): 0.21844204
percentage of variance explained: 0.704
shape of reconstructed matrix:  (6040, 3706)<jupyter_text>We also create a method predict for predicting ratings that a user might give to a movie he hasn't seen yet, and a method recommend for recommending movies for a given user. Both methods take in a de-biased matrix and the bias to apply to the final results.<jupyter_code>def predict(user_id, movie_ids, X_rec, bias, 
            users_id2idx, movies_id2idx, movies_id2name):
    predictions = []
    for movie_id in sorted(movie_ids):
        user_idx = users_id2idx[user_id]
        movie_idx = movies_id2idx[movie_id]
        movie_name = movies_id2name[movie_id]
        prediction = bias + X_rec[user_idx, movie_idx]
        predictions.append((user_id, movie_id, movie_name, prediction)) #append the prediction rate of the movies against the userId
    return predictions
def recommend(user_id, X_rec, top_n, bias, 
              users_id2idx, movies_idx2id, movies_id2name):
    user_idx = users_id2idx[user_id]
    rec_movies_idxs = np.argsort(-1 * X_rec[user_idx])[0:top_n] #recommendating top n movies
    recommendations = []
    for movie_idx in rec_movies_idxs:
        movie_id = movies_idx2id[movie_idx]
        movie_name = movies_id2name[movie_id]
        pred_rating = bias + X_rec[user_idx, movie_idx]
        recommendations.append((user_id, movie_id, movie_name, pred_rating)) # append all top n recommending movies against the userId
    return recommendations<jupyter_output><empty_output><jupyter_text>Here by using SVD factorization method we are predicting the rating for userId 215 and on that basis top 10 movies are recommended to that userId.<jupyter_code>R_rec, _, _ = reconstruct_using_svd(Rui_val, 10)
movie_ids = User_Movie_notpred(215,ratings_df,10)
preds = predict(215, movie_ids, R_rec, bg + bu + bui, users_id2idx, 
                movies_id2idx, movies_id2name) 
for pred in preds:
    print(pred)
print("---------------------------------------------------------------------")
R_rec, _, _ = reconstruct_using_svd(Ri_val, 10)
recs = recommend(215, R_rec, 10, bg + bi, users_id2idx, movies_idx2id, movies_id2name)
for rec in recs:
    print(rec)<jupyter_output>(215, 594, 'Snow White and the Seven Dwarfs (1937)', 3.7326605)
(215, 595, 'Beauty and the Beast (1991)', 3.7849703)
(215, 661, 'James and the Giant Peach (1996)', 3.7088013)
(215, 914, 'My Fair Lady (1964)', 3.725639)
(215, 919, 'Wizard of Oz, The (1939)', 3.8021135)
(215, 1193, "One Flew Over the Cuckoo's Nest (1975)", 3.7281618)
(215, 1287, 'Ben-Hur (1959)', 3.77651)
(215, 2355, "Bug's Life, A (1998)", 3.7777786)
(215, 2804, 'Christmas Story, A (1983)', 3.8005369)
(215, 3408, '<NAME> (2000)', 3.7638147)
---------------------------------------------------------------------
(215, 260, 'Star Wars: Episode IV - A New Hope (1977)', 4.0347776)
(215, 1210, 'Star Wars: Episode VI - Return of the Jedi (1983)', 4.0347443)
(215, 1198, 'Raiders of the Lost Ark (1981)', 4.0149407)
(215, 480, 'Jurassic Park (1993)', 4.012251)
(215, 1196, 'Star Wars: Episode V - The Empire Strikes Back (1980)', 4.005173)
(215, 110, 'Braveheart (1995)', 3.9968603)
(215, 589, 'Terminator 2: Judgment Day (19[...]<jupyter_text>SVD++ Algorithm While SVD is able to provide very good recommendation quality, its ability to use only explicit numerical ratings as user-items interactions constitutes a limitation. Modern day recommender systems should exploit all available interactions both explicit (e.g. numerical ratings) and implicit (e.g. likes, purchases, skipped, bookmarked). To this end SVD++ was designed to take into account implicit interactions as wellSVD++ has however some disadvantages, with the main drawback being that this method is not model-based. This means that if a new user is added, the algorithm is incapable of modeling it unless the whole model is retrained. Even though the system might have gathered some interactions for that new user, it's latent factors are not available and therefore no recommendations can be computed. This is an example of a cold-start problem, that is the recommender cannot deal efficiently with new users or items and specific strategies should be put in place to handle this disadvantage.<jupyter_code># Use movielens 1m dataset from Dataset package
from surprise import Dataset
data = Dataset.load_builtin('ml-1m')
type(data)
# Split train and test dataset
trainset, testset = train_test_split(data, test_size=.15)
svdpp = SVDpp(n_factors=20, n_epochs=20, lr_all=0.007, reg_all=0.02)
svdpp.fit(trainset)
test_pred = svdpp.test(testset)
print("SVDpp : Test Set")
accuracy.mae(test_pred, verbose=True)<jupyter_output>SVDpp : Test Set
MAE:  0.6697<jupyter_text>We have explored and used Surprise package: This package has been specially developed to make recommendation based on collaborative filtering easy. It has default implementation for a variety of CF algorithms.  Hybrid Model (Content Based Filtering + SVD) Content-based filtering, also referred to as cognitive filtering, recommends items based on a comparison between the content of the items and a user profile. The content of each item is represented as a set of descriptors or terms, typically the words that occur in a document. The user profile is represented with the same terms and built up by analyzing the content of items which have been seen by the user. The concepts of Term Frequency (TF) and Inverse Document Frequency (IDF) are used in information retrieval systems and also content based filtering mechanisms (such as a content based recommender). They are used to determine the relative importance of a document / article / news item / movie etc.We will consider genres as an important parameter to recommend user the movie he watches based on generes of movie user has already watched. For calculating distances, many similarity coefficients can be calculated. Most widely used similarity coefficients are Euclidean, Cosine, Pearson Correlation etc. Cosine similarity Well cosine similarity is a measure of similarity between two non zero vectors. Cosine similarity finds how two vectors are related to each other using measuring cosine angle between these vectors. ![image.png](attachment:image.png) We will use cosine distance here. Here we are insterested in similarity. That means higher the value more similar they are. But as the function gives us the distance, we will deduct it from 1.<jupyter_code># Define a TF-IDF Vectorizer Object.
tfidf_movies_genres = TfidfVectorizer(token_pattern = '[a-zA-Z0-9\-]+')

#Replace NaN with an empty string
movies_df['genres'] = movies_df['genres'].replace(to_replace="(no genres listed)", value="")

#Construct the required TF-IDF matrix by fitting and transforming the data
tfidf_movies_genres_matrix = tfidf_movies_genres.fit_transform(movies_df['genres'])
cosine_sim_movies = linear_kernel(tfidf_movies_genres_matrix, tfidf_movies_genres_matrix)<jupyter_output><empty_output><jupyter_text>Here a function is defined to calculate the top 2 movies to recommend based on given movie title genres.cosine_sim_movies = the cosine similarities between moviesWe are returning the title of the movie as recommendation.<jupyter_code>def get_recommendations_based_on_genres(movie_title, cosine_sim_movies=cosine_sim_movies):
    
    # Get the index of the movie that matches the title
    idx_movie = movies_df.loc[movies_df['title'].isin([movie_title])]
    idx_movie = idx_movie.index
    
    # Get the pairwsie similarity scores of all movies with that movie
    sim_scores_movies = list(enumerate(cosine_sim_movies[idx_movie][0]))
    
    # Sort the movies based on the similarity scores
    sim_scores_movies = sorted(sim_scores_movies, key=lambda x: x[1], reverse=True)

    # Get the scores of the 10 most similar movies
    sim_scores_movies = sim_scores_movies[1:3]
    
    # Get the movie indices
    movie_indices = [i[0] for i in sim_scores_movies]
    
    # Return the top 2 most similar movies
    return movies_df['title'].iloc[movie_indices]
get_recommendations_based_on_genres("Friday (1995)")<jupyter_output><empty_output><jupyter_text>Define a function to calculate top movies to be recommended to the user based on the movie user has wateched.<jupyter_code>def get_recommendation_content_model(userId):
    
    recommended_movie_list = []
    movie_list = []
    df_rating_filtered = ratings_df[ratings_df["userId"]== userId]
    for key, row in df_rating_filtered.iterrows():
        movie_list.append((movies_df["title"][row["movieId"]==movies_df["movieId"]]).values) 
    for index, movie in enumerate(movie_list):
        for key, movie_recommended in get_recommendations_based_on_genres(movie[0]).iteritems():
            recommended_movie_list.append(movie_recommended)

    # removing already watched movie from recommended list    
    for movie_title in recommended_movie_list:
        if movie_title in movie_list:
            recommended_movie_list.remove(movie_title)
    
    return set(recommended_movie_list)
get_recommendation_content_model(1)<jupyter_output><empty_output><jupyter_text>Hybrid ModelA hybrid approach, combining collaborative filtering and content-based filtering could be more effective in some cases. Hybrid approaches can be implemented in several ways: by making content-based and collaborative-based predictions separately and then combining them; by adding content-based capabilities to a collaborative-based approach (and vice versa); or by unifying the approaches into one model<jupyter_code>svd = SVD()
trainset = data.build_full_trainset()
svd.train(trainset)
#movie=movies_df
#rating=ratings_df
def hybrid_content_svd_model(userId):
    """
    hydrid the functionality of content based and svd based model to recommend user top 10 movies. 
    :param userId: userId of user
    :return: list of movies recommended with rating given by svd model
    """
    recommended_movies_by_content_model = get_recommendation_content_model(userId)
    recommended_movies_by_content_model = movies_df[movies_df.apply(lambda movie: movie["title"] in recommended_movies_by_content_model, axis=1)]
    for key, columns in recommended_movies_by_content_model.iterrows():
        predict = svd.predict(userId, columns["movieId"])
        recommended_movies_by_content_model.loc[key, "svd_rating"] = predict.est
        
    return recommended_movies_by_content_model.sort_values("svd_rating", ascending=False).iloc[0:11]
recommended_movies_by_content_model = get_recommendation_content_model(215)
hybrid_content_svd_model(215)<jupyter_output>C:\Users\Akshay\PycharmProjects\Big_Data_Class\venv\lib\site-packages\pandas\core\indexing.py:362: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[key] = _infer_fill_value(value)
C:\Users\Akshay\PycharmProjects\Big_Data_Class\venv\lib\site-packages\pandas\core\indexing.py:543: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  self.obj[item] = s<jupyter_text>ConclusionWith deep learning algorithms used in this notebook, We have implemented Restricted Boltzmann Machine, Deep Autoencoder, and simple deep neural network for this project to analyze movielens dataset. The Restricted Boltzmann Machine model has two layers of neurons, one of which is what we call a visible input layer and the other is called a hidden layer. The hidden layer is used to learn features from the information fed through the input layer. For our model, the input is going to contain X neurons, where X is the number of movies in our dataset. Taking hiddenUnits = 20, error_function = MAE, epochs = 15 and activation function = {tf.sigmoid and tf.relu }, the model showed MAE of 0.0449. Deep Autoencoder algorithms used to predict the rating for user who has not rated movies. Deep Autoencoder algorithm predicted movie ratings with epochs=100, batch size= 100 and MAE as 3.37 for test and 3.05 for train phase. Simple deep neural network takes user and movies as separate matrix and create embeddings on both matrices after that feature creation has done by taking dot product of the user and the item embeddings to feed in the network. Network has trained to predict the user rating for movie with MAE 0.69. Using traditional algorithms, We attempted to build a model-based Collaborative Filtering movie recommendation system based on latent features from a low rank matrix factorization method called SVD and SVD++.For SVD model we get an MAE of 0.218 with bias and 0.064 without bias and for SVD++ model we get an MAE of 0.744. After analyzing results from implemented model, We have found that Restricted Boltzmann Machine model performed better than other models due to the restriction over the structure of Boltzmann Machine.Also, User-based Collaborative Filtering gives recommendations that can be complements to the item the user was interacting with. This might be a stronger recommendation than what a item-based recommender can provide as users might not be looking for direct substitutes to a movie they had just viewed or previously watched.<jupyter_code>header_Cl = ["Model" , "Mean Absoulte Error"]
table_cl  = [
        ["RBM",'0.0449'],
        ["Autoencoders",'3.37'],
        ["SVD",'0.218 with bias and 0.064 without bias'],
        ["SVD++",'0.744'],
        ["DNN with Keras",'0.69'],
            ]
print(tabulate(table_cl, header_Cl, tablefmt="fancy_grid"))<jupyter_output>╒════════════════╤════════════════════════════════════════╕
│ Model          │ Mean Absoulte Error                    │
╞════════════════╪════════════════════════════════════════╡
│ RBM            │ 0.0449                                 │
├────────────────┼────────────────────────────────────────┤
│ Autoencoders   │ 3.37                                   │
├────────────────┼────────────────────────────────────────┤
│ SVD            │ 0.218 with bias and 0.064 without bias │
├────────────────┼────────────────────────────────────────┤
│ SVD++          │ 0.744                                  │
├────────────────┼────────────────────────────────────────┤
│ DNN with Keras │ 0.69                                   │
╘════════════════╧════════════════════════════════════════╛
{'fraction_non_alphanumeric': 0.09454976742462148, 'fraction_numerical': 0.03572950834862215, 'mean_word_length': 5.165676416819013, 'pattern_counts': {'":': 3, '<': 176, '<?xml version=': 0, '>': 177, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 14, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '46', 'dir': 'jupyter-structured-clean-dedup', 'id': '523389', 'license': 'MIT', 'n_tokens_mistral': 17833, 'n_tokens_neox': 16031, 'n_words': 5637, 'path': 'Final_Project/Portfolio/Movie_Recommendation_using_DeepLearning.ipynb', 'repo_name': 'axxypatel/Big_Data_Class', 'size': '53970'}