<jupyter_start><jupyter_text>Collaboratibe Filtering (Item-Item) : Movie Recommendation System We are creating a collaborative filtering based movie recommendatipon system where are trying to create user-movie rating matrix, where ratings given by users are playing features/ patterns role and based on nearest neighbours algorithm finding closest movies based on those patterns. Kind of finding similiar movies/ item-item similarity based recommendation system.  References https://www.geeksforgeeks.org/recommendation-system-in-python/<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns<jupyter_output><empty_output><jupyter_text>Loading data  Ratings<jupyter_code>ratings = pd.read_csv("./datasets/ratings.csv")
ratings.head()<jupyter_output><empty_output><jupyter_text>Movies<jupyter_code>movies = pd.read_csv("./datasets/movies.csv")
movies.head()<jupyter_output><empty_output><jupyter_text>General Shapes Information<jupyter_code>print(f"""
ratings dataframe shape :{ratings.shape}
movies dataframe shape  :{movies.shape}

total movies : {movies.movieId.unique().shape}
total rated movies : {ratings.movieId.unique().shape}

total users : {ratings.userId.unique().shape}
All kind of ratings given by users : {ratings.rating.unique()}""")<jupyter_output>ratings dataframe shape :(100836, 4)
movies dataframe shape  :(9742, 3)

total movies : (9742,)
total rated movies : (9724,)

total users : (610,)
All kind of ratings given by users : [4.  5.  3.  2.  1.  4.5 3.5 2.5 0.5 1.5]<jupyter_text>Get unique ids<jupyter_code>all_user_ids = ratings.userId.unique()
all_movie_ids = movies.movieId.unique()<jupyter_output><empty_output><jupyter_text>Creating Maps  Map to convert user id to user mapping id<jupyter_code>user_id_maps = { i[0]:i[1] for i in enumerate(all_user_ids) }<jupyter_output><empty_output><jupyter_text>Map to convert user mapping id to user id<jupyter_code>user_id_inv_maps = { user_id_maps[i]:i for i in user_id_maps }<jupyter_output><empty_output><jupyter_text>Map to convert movie id to movie mapping id<jupyter_code>movie_id_maps = { i[0]:i[1] for i in enumerate(all_movie_ids) }<jupyter_output><empty_output><jupyter_text>Map to convert movie mapping id to movie id<jupyter_code>movie_id_inv_maps = { movie_id_maps[i]:i for i in movie_id_maps }<jupyter_output><empty_output><jupyter_text>user-movie average rating<jupyter_code>user_movie_ratings_df = ratings[['userId','movieId','rating']]\
                            .groupby(by=['userId', 'movieId'],as_index=False)\
                            .mean()
user_movie_ratings_df.columns
user_movie_ratings_df.head()<jupyter_output><empty_output><jupyter_text>Converting existing ids to mapping id<jupyter_code>user_movie_ratings_df['user_maps'] = user_movie_ratings_df['userId'].apply(lambda x: user_id_inv_maps[x])
user_movie_ratings_df['movie_maps'] = user_movie_ratings_df['movieId'].apply(lambda x: movie_id_inv_maps[x])
user_movie_ratings_df.head()<jupyter_output><empty_output><jupyter_text>Creating User-Movie Rating Matrix<jupyter_code>from scipy.sparse import csr_matrix
user_maps = user_movie_ratings_df.user_maps.values
movie_maps = user_movie_ratings_df.movie_maps.values
rating_values = user_movie_ratings_df.rating.values

rating_matrix = csr_matrix((rating_values, (movie_maps, user_maps))).toarray()
rating_matrix<jupyter_output><empty_output><jupyter_text>So How is this working actually```             user_maps = [0 1 2 3 4 5 6 7 8 9 10]    movie_maps = [0 1 2 3 4 5 6 7 8 9 10]    ratings = [3 4 1 5 5 2 5 1 1 4]                                    Users maps                                                 0  1  2  3  4  5  6  7  8  9  10                         _  _  _  _  _  _  _  _  _  _  _                       0 |3  0  0  0  0  0  0  0                       1 |0  4  0  0  0  0  0           movies maps 2 |0  0  1  0  0  0  .                      3 |0  0  0  5  0  0  .                              4 |0  0  0  0  5  0  .                      5 |0  0  0  0  0  2                       6 |0  0  0  0  0  0                        7 |.  .  .  .                        8 |.  . ```<jupyter_code>rating_matrix.shape<jupyter_output><empty_output><jupyter_text>Clustering Model<jupyter_code>from sklearn.neighbors import NearestNeighbors
n_neighbors = 10
metric = 'cosine'
kNN = NearestNeighbors(n_neighbors= n_neighbors, algorithm='brute', metric=metric)
kNN.fit(rating_matrix)<jupyter_output><empty_output><jupyter_text>Testing Model<jupyter_code>movie_id = 1
movies.query(f'movieId == {movie_id}')
input_movie_vector = rating_matrix[movie_id_inv_maps[movie_id]].reshape(1,-1)
print("Input Movie Vector:",input_movie_vector.shape)

distances, collected_neighbours = kNN.kneighbors(
    X=input_movie_vector, n_neighbors=n_neighbors, return_distance=True)

iter_map = map(lambda x: movie_id_maps[x], collected_neighbours[0])

recoms = movies[movies.movieId.isin(iter_map)].reset_index()

recoms['distances'] = distances[0]

print("\nRecommendations :")
recoms.head(n_neighbors)<jupyter_output>Input Movie Vector: (1, 610)

Recommendations :<jupyter_text>Plotting distances<jupyter_code>fig, ax = plt.subplots(1,1, figsize=(10,5))
sns.barplot(data=recoms, y='title', x='distances', orient='h')
plt.show()<jupyter_output><empty_output><jupyter_text>Recommendation System Modelling Class<jupyter_code>from utils.engine import BasicMovieRecommedation
from utils.plot_utils import plot_recommendations
model = BasicMovieRecommedation(n_neighbors=20)
model.fit(rating_df=ratings[['userId','movieId','rating']], movie_df=movies[['movieId']])
movie_input = 1
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)
movie_input = 50
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)
movie_input = 605
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)
movie_input = 999
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)

movie_input = 189333
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)

movie_input = 187593
print(movies[movies.movieId.isin([movie_input])])
movie_ids, distances = model.get_recommendations(movie_input, 5)

recommendations = movies[movies.movieId.isin(movie_ids)].reset_index()
recommendations['distances'] = distances

plot_recommendations(recommendations)<jupyter_output>movieId              title                genres
9709   187593  Deadpool 2 (2018)  Action|Comedy|Sci-Fi
{'fraction_non_alphanumeric': 0.10675293166194905, 'fraction_numerical': 0.027227389136002155, 'mean_word_length': 4.974235104669887, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 66, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 11, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '144730', 'license': 'MIT', 'n_tokens_mistral': 2829, 'n_tokens_neox': 2557, 'n_words': 504, 'path': 'notebooks/cf_item_item.ipynb', 'repo_name': 'NishantBaheti/Recommendation-Systems', 'size': '7419'}