<jupyter_start><jupyter_text>Assignment:  Book Rental Recommendation*The comments/sections provided are your cues to perform the assignment. You don't need to limit yourself to the number of rows/cells provided. You can add additional rows in each section to add more lines of code.**If at any point in time you need help on solving this assignment, view our demo video to understand the different steps of the code.***Happy coding!*** * *  Book Rental Recommendation__DESCRIPTION__BookRent is the largest online and offline book rental chain in India. The company charges a fixed rental fee for a book per month. Lately, the company has been losing its user base.The main reason for this is that users are not able to choose the right books for themselves. The company wants to solve this problem and increase its revenue and profit.__Objective:__ You, as an ML expert, have to model a recommendation engine so that users get recommendations for books based on the behavior of similar users. This will ensure that users are renting books based on their individual tastes.__Actions to Perform:__* Read the books dataset and explore it.* Clean up NaN values.* Read the data where ratings are given by users.* Take a quick look at the number of unique users and books.* Convert ISBN to numeric numbers in the correct order.* Do the same for user_id. Convert it into numeric order.* Convert both user_id and ISBN to the ordered list i.e. from 0...n-1.* Re-index columns to build matrix later on.* Split your data into two sets (training and testing).* Calculate the cosine similarity.* Use the evaluation metrics to make predictions.<jupyter_code>#Import required libraries
import numpy as np
import pandas as pd
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Read the datasets and explore it<jupyter_code>#Read the data using pandas in DataFrame
df_users = pd.read_csv('BX-Users.csv',encoding='latin-1')
df_users.head()
df_users.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 278859 entries, 0 to 278858
Data columns (total 3 columns):
 #   Column    Non-Null Count   Dtype  
---  ------    --------------   -----  
 0   user_id   278859 non-null  object 
 1   Location  278858 non-null  object 
 2   Age       168096 non-null  float64
dtypes: float64(1), object(2)
memory usage: 4.3+ MB<jupyter_text>The Age column in df_users dataset has number of NaN values, which can be dropped. since we have a large set of dataset (168096 rows) with non-null values in Age column. we can drop entire set of rows with NaN values.<jupyter_code>df_users.shape
#Read the books Data and explore
#column_names = ['isbn', 'book_title']
df_books = pd.read_csv('BX-Books.csv', encoding='latin-1')
df_books.head()
df_books.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 271379 entries, 0 to 271378
Data columns (total 5 columns):
 #   Column               Non-Null Count   Dtype 
---  ------               --------------   ----- 
 0   isbn                 271379 non-null  object
 1   book_title           271379 non-null  object
 2   book_author          271378 non-null  object
 3   year_of_publication  271379 non-null  object
 4   publisher            271377 non-null  object
dtypes: object(5)
memory usage: 5.2+ MB<jupyter_text>The book_author and publisher columns in df_books dataset has number of NaN values. Since we have a large set of dataset NaN rows can be dropped. .<jupyter_code>df_books.shape<jupyter_output><empty_output><jupyter_text>Read the data where ratings are given by users We have taken only first 5000 rows of ratings given by users. Otherwise, Out Of Memory error can occur.<jupyter_code>df_ratings = pd.read_csv('BX-Book-Ratings.csv',encoding='latin-1',nrows=5000)
df_ratings.head()
df_ratings.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 5000 entries, 0 to 4999
Data columns (total 3 columns):
 #   Column   Non-Null Count  Dtype 
---  ------   --------------  ----- 
 0   user_id  5000 non-null   int64 
 1   isbn     5000 non-null   object
 2   rating   5000 non-null   int64 
dtypes: int64(2), object(1)
memory usage: 97.7+ KB<jupyter_text>Clean up NaN valuesDropping rows with NaN values in both df_users and df_books DataFrames.<jupyter_code>#Clean up NaN values in df_users
df_users = df_users.dropna(subset=['Age'])
df_users.info()
#Clean up NaN values in df_books
df_books = df_books.dropna(subset=['book_author','publisher'])
df_books.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 271376 entries, 0 to 271378
Data columns (total 5 columns):
 #   Column               Non-Null Count   Dtype 
---  ------               --------------   ----- 
 0   isbn                 271376 non-null  object
 1   book_title           271376 non-null  object
 2   book_author          271376 non-null  object
 3   year_of_publication  271376 non-null  object
 4   publisher            271376 non-null  object
dtypes: object(5)
memory usage: 7.2+ MB<jupyter_text>Merging the df_ratings and df_books dataframes based on 'isbn', to form a new combined DataFrame 'df'.<jupyter_code>df = pd.merge(df_ratings,df_books,on='isbn')
df.head()
df.shape<jupyter_output><empty_output><jupyter_text>Take a quick look at the number of unique users and books<jupyter_code>n_users = df.user_id.nunique()
n_books = df.isbn.nunique()

print('Num. of Users: '+ str(n_users))
print('Num of Books: '+str(n_books))<jupyter_output>Num. of Users: 591
Num of Books: 4122<jupyter_text>Convert ISBN to numeric numbers in the correct order<jupyter_code>isbn_list = df.isbn.unique()
print(" Length of isbn List:", len(isbn_list))
def get_isbn_numeric_id(isbn):
    #print ("  isbn is:" , isbn)
    itemindex = np.where(isbn_list==isbn)
    return itemindex[0][0]<jupyter_output>Length of isbn List: 4122<jupyter_text>Convert user_id to numeric numbers in the correct order<jupyter_code>userid_list = df.user_id.unique()
print(" Length of user_id List:", len(userid_list))
def get_user_id_numeric_id(user_id):
    #print ("  isbn is:" , isbn)
    itemindex = np.where(userid_list==user_id)
    return itemindex[0][0]<jupyter_output>Length of user_id List: 591<jupyter_text>Convert both user_id and ISBN to the ordered list i.e. from 0...n-1<jupyter_code>df['user_id_order'] = df['user_id'].apply(get_user_id_numeric_id)
df['isbn_id'] = df['isbn'].apply(get_isbn_numeric_id)
df.head()<jupyter_output><empty_output><jupyter_text>Re-index columns to build matrix later on<jupyter_code>new_col_order = ['user_id_order', 'isbn_id', 'rating', 'book_title', 'book_author',
                 'year_of_publication','publisher','isbn','user_id']
df = df.reindex(columns= new_col_order)
df.head()<jupyter_output><empty_output><jupyter_text>Split data into two sets (training and testing)Recommendation Systems  are difficult to evaluate, but we will still learn how to evaluate them. In order to do this, we'll split our data into two sets. However, we won't do our classic X_train,X_test,y_train,y_test split. Instead, we can actually just segement the data into two sets of data:<jupyter_code>from sklearn.model_selection import train_test_split
train_data, test_data = train_test_split(df, test_size=0.30)<jupyter_output><empty_output><jupyter_text>__Memory-Based Collaborative Filtering__Memory-Based Collaborative Filtering approaches can be divided into two main sections: **user-item filtering** and **item-item filtering**. A *user-item filtering* will take a particular user, find users that are similar to that user based on similarity of ratings, and recommend items that those similar users liked. In contrast, *item-item filtering* will take an item, find users who liked that item, and find other items that those users or similar users also liked. It takes items as input and outputs other items as recommendations. * *Item-Item Collaborative Filtering*: “Users who liked this item also liked …”* *User-Item Collaborative Filtering*: “Users who are similar to you also liked …” In both cases, we create a user-book matrix which is built from the entire dataset.Since we have split the data into testing and training, we will need to create two ``[828 x 8051]`` matrices (all users by all books). This is going to be  a very large matrixThe training matrix contains 70% of the ratings and the testing matrix contains 30% of the ratings.<jupyter_code>#Create two user-book matrices, one for training and another for testing
train_data_matrix = np.zeros((n_users, n_books))
for line in train_data.itertuples():
    train_data_matrix[line[1]-1, line[2]-1] = line[3]  

test_data_matrix = np.zeros((n_users, n_books))
for line in test_data.itertuples():
    test_data_matrix[line[1]-1, line[2]-1] = line[3]<jupyter_output><empty_output><jupyter_text>Calculate the cosine similarity.we can use the [pairwise_distances](http://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.pairwise_distances.html) function from sklearn to calculate the cosine similarity. Note, the output will range from 0 to 1 since the ratings are all positive.<jupyter_code>from sklearn.metrics.pairwise import pairwise_distances
user_similarity = pairwise_distances(train_data_matrix, metric='cosine')
item_similarity = pairwise_distances(train_data_matrix.T, metric='cosine')
user_similarity
item_similarity<jupyter_output><empty_output><jupyter_text>Use the evaluation metrics to make predictionsThere are many evaluation metrics, but one of the most popular metric used to evaluate accuracy of predicted ratings is *Root Mean Squared Error (RMSE)*.<jupyter_code>def predict(ratings, similarity, type='user'):
    if type == 'user':
        mean_user_rating = ratings.mean(axis=1)
        #use np.newaxis so that mean_user_rating has same format as ratings
        ratings_diff = (ratings - mean_user_rating[:, np.newaxis]) 
        pred = mean_user_rating[:, np.newaxis] + similarity.dot(ratings_diff) / np.array([np.abs(similarity).
                                                                                          sum(axis=1)]).T
    elif type == 'item':
        pred = ratings.dot(similarity) / np.array([np.abs(similarity).sum(axis=1)])     
    return pred
item_prediction = predict(train_data_matrix, item_similarity, type='item')
user_prediction = predict(train_data_matrix, user_similarity, type='user')<jupyter_output><empty_output><jupyter_text>Since, we only want to consider predicted ratings that are in the test dataset, we filter out all other elements in the prediction matrix with: `prediction[ground_truth.nonzero()]`.<jupyter_code>from sklearn.metrics import mean_squared_error
from math import sqrt
def rmse(prediction, ground_truth):
    prediction = prediction[ground_truth.nonzero()].flatten() 
    ground_truth = ground_truth[ground_truth.nonzero()].flatten()
    return sqrt(mean_squared_error(prediction, ground_truth))
print('User-based CF RMSE: ' + str(rmse(user_prediction, test_data_matrix)))
print('Item-based CF RMSE: ' + str(rmse(item_prediction, test_data_matrix)))<jupyter_output>User-based CF RMSE: 7.725507960248443
Item-based CF RMSE: 7.722860869362833
{'fraction_non_alphanumeric': 0.09405180973012005, 'fraction_numerical': 0.025724343352288112, 'mean_word_length': 4.360425737784229, 'pattern_counts': {'":': 0, '<': 65, '<?xml version=': 0, '>': 65, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '640433', 'license': 'MIT', 'n_tokens_mistral': 3596, 'n_tokens_neox': 3343, 'n_words': 1169, 'path': '10.17_Book_Rental_Recommendation/Book Rental Recommendation.ipynb', 'repo_name': 'mohamedsuhaib/Machine_Learning_Practice_Projects', 'size': '11079'}