<jupyter_start><jupyter_text>Movie Recommender  Problem Statement: - **Build a movie Recommender System which can recommender some movie's based on user preference history.**- **Tasks performed:** - 1) Create recommender system using ratings.csv file.  - 2)	Display predicted rating for a particular userID and MovieID combinations (both taken as keyboard input) - 3)	Recommend a user to watch top 10 movies, which has not watched yet (as per his rating predictions). Take userID as             a keyboard input. Fix the threshold rating as 2.5 - 4)	Display the MovieID, IMDB ID, Average User Rating (excluding predictions), genres and tag of all the movies found               in Step 3 as a data frame.  import Libraries<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Loading All the Datasets<jupyter_code>tag = pd.read_csv("tags.csv")
tag.head()
links = pd.read_csv("links.csv")
links.head()
ratings = pd.read_csv("ratings.csv")
ratings.head()
movies = pd.read_csv("movies.csv")
movies.head()<jupyter_output><empty_output><jupyter_text>Let's explore the rating dataset on which we will build the Recommendation system<jupyter_code>ratings.head()
ratings.shape<jupyter_output><empty_output><jupyter_text>- **We can say that the rating dataset is having 100836 observations and 4 features**<jupyter_code>ratings.info()
ratings['rating'].unique()<jupyter_output><empty_output><jupyter_text>- **So here the rating scale is between 0 to 5.**<jupyter_code>ratings['rating'].min()<jupyter_output><empty_output><jupyter_text>- **minumum rating is 0.5**<jupyter_code>ratings['rating'].max()<jupyter_output><empty_output><jupyter_text>- **maximum rating is 5**<jupyter_code>ratings['rating'].describe()
ratings['rating'].value_counts()<jupyter_output><empty_output><jupyter_text>- **By seeing count we can say that 4 rating is given by most user, than rating 3 and so on.**  Data Visualization<jupyter_code>import seaborn as sns
sns.boxplot(ratings['rating'])<jupyter_output><empty_output><jupyter_text>- **By plotting boxplot we can say thier are very less outliers say, two.**<jupyter_code>sns.distplot(ratings['rating'])<jupyter_output><empty_output><jupyter_text>- **By seeing this graph we can say that 4 rating is given by most user, than rating 3 and so on.**  Checking null values<jupyter_code>ratings.isnull().sum()
import seaborn as sns
sns.heatmap(ratings.isnull())<jupyter_output><empty_output><jupyter_text>- **There is no null values in this dataset.**  Importing some of the libraries which is required to build the recommedation system from the Surprise package<jupyter_code>from surprise import KNNWithMeans
from surprise import Dataset
from surprise import accuracy
from surprise import Reader
from surprise.model_selection import train_test_split
ratings.head()
ratings['userId'].count()
plt.hist(ratings['userId'])
ratings['rating'].hist(bins=60)
ratings['userId'].hist(bins=20)
ratings.groupby("userId")[['userId']].count()<jupyter_output><empty_output><jupyter_text>Let's build the Recommender System Now using ratings dataset<jupyter_code>## loading the dataset with the required column
reader = Reader(rating_scale =(0,5))
data = Dataset.load_from_df(ratings[['userId','movieId','rating']], reader)
## splitting dataset into training and testing datasets
[trainset,testset] = train_test_split(data, test_size = 0.15, shuffle = True)
## Find similarity using KNN and cosine similarity formalua 
recom = KNNWithMeans(k=50, sim_options = {'name':'cosine','user_based': True})
recom.fit(data.build_full_trainset())       # using full training set and
test_pred = recom.test(testset)
RMSE = accuracy.rmse(test_pred)

#test_pred<jupyter_output><empty_output><jupyter_text>Taking Input From the User from KeyBoard & Displaying predicted rating for a particular userID and MovieID combinations<jupyter_code>userId = int(input("Enter userId: "))
movieId = int(input("Enter movieId: "))
print("The rating value for  userID:",userId, "and", "movieID:",movieId, "is :",round((recom.predict(userId,movieId)[3]),3))<jupyter_output>Enter userId: 7
Enter movieId: 8
The rating value for  userID: 7 and movieID: 8 is : 2.611<jupyter_text>=================================================================  Merging ratings and movies datasets<jupyter_code>movie_data = pd.merge(ratings, movies, on='movieId')
movie_data.head()
import numpy as np
from scipy.sparse import csr_matrix         # compressed sparsh matrix
from sklearn.neighbors import NearestNeighbors 
import sklearn
user_movie_rating_1= movie_data.pivot_table(index='title', columns='userId', values='rating').fillna(0)
user_movie_rating_1.head()
movie_data_matrix = csr_matrix(user_movie_rating_1.values)
movie_data_matrix
model_knn = NearestNeighbors(metric = 'cosine', algorithm='brute')
model_knn.fit(movie_data_matrix)                                   # here .fit will take only compressed_matrix and we
                                                                  # cannot pass dataframe and fit will calculate distance here<jupyter_output><empty_output><jupyter_text>Similar Movies for Movie 1<jupyter_code>query_index1 = 0    # 1st movie
distances, indices = model_knn.kneighbors(
                     user_movie_rating_1.iloc[query_index1, :].values.reshape(1,-1),
                      n_neighbors = 11)

print('Similar Movies for {0}\n'.format(user_movie_rating_1.index[query_index1]))
for i in indices.flatten()[1:]:
    print(user_movie_rating_1.index[i])<jupyter_output>Similar Movies for '71 (2014)

Invincible Shaolin (1978)
Digimon: The Movie (2000)
Digging Up the Marrow (2014)
Tokyo! (2008)
Tournament, The (2009)
Ginger Snaps: Unleashed (2004)
Robin-B-Hood (Bo bui gai wak) (2006)
Little Big Soldier (Da bing xiao jiang) (2010)
Ginger Snaps Back: The Beginning (2004)
Beyond Re-Animator (2003)<jupyter_text>Similar Movies for Movie 2<jupyter_code>query_index2 = 1    # 2nd movie
distances, indices = model_knn.kneighbors(
                     user_movie_rating_1.iloc[query_index2, :].values.reshape(1,-1),
                      n_neighbors = 11)

print('Similar Movies for {0}\n'.format(user_movie_rating_1.index[query_index2]))
for i in indices.flatten()[1:]:
    print(user_movie_rating_1.index[i])<jupyter_output>Similar Movies for 'Hellboy': The Seeds of Creation (2004)

Space Battleship Yamato (2010)
'Hellboy': The Seeds of Creation (2004)
All the Right Moves (1983)
Hidden Fortress, The (Kakushi-toride no san-akunin) (1958)
...And Justice for All (1979)
'Round Midnight (1986)
Kagemusha (1980)
Sanjuro (Tsubaki Sanjûrô) (1962)
Ghost Rider: Spirit of Vengeance (2012)
Throne of Blood (Kumonosu jô) (1957)<jupyter_text>Similar Movies for Movie 3<jupyter_code>query_index3 = 2    # 3rd movie
distances, indices = model_knn.kneighbors(
                     user_movie_rating_1.iloc[query_index3, :].values.reshape(1,-1),
                      n_neighbors = 11)

print('Similar Movies for {0}\n'.format(user_movie_rating_1.index[query_index3]))
for i in indices.flatten()[1:]:
    print(user_movie_rating_1.index[i])<jupyter_output>Similar Movies for 'Round Midnight (1986)

Angry Red Planet, The (1959)
American Friend, The (Amerikanische Freund, Der) (1977)
Asphyx, The (1973)
Annie Get Your Gun (1950)
Agony and the Ecstasy, The (1965)
Atragon (Kaitei Gunkan) (1963)
Amazing Transparent Man, The (1960)
Attack of the Crab Monsters (1957)
And Starring <NAME> as Himself (2003)
'Hellboy': The Seeds of Creation (2004)<jupyter_text>---  Recommend a user to watch top 10 movies, which has not watched yet (as per his rating predictions). Take userID as a keyboard input. Fix the threshold rating as 2.5<jupyter_code>movie_lst = {} # Creating empty dictionary to append userid as key and movie id is value

user_id = int(input("Enter the user id: "))
for movie_id in range(1,193609):
    if recom.predict(user_id,movie_id)[3] > 2.5:
        movie_lst[movie_id] = recom.predict(user_id,movie_id)[3]

# finding 10 highest values in a Dictionary  
  
from collections import Counter
k = Counter(movie_lst)

high = k.most_common(10)
print("\n")

#printing those 10 recomended movie
print("Recommonded Top 10 movies for userId {0} are listed below\n".format(user_id))



# suuggesting 10  movies 
for i in range(10):
    print(i+1,"-->",list(movies[movies.movieId==high[i][0]]["title"])[0])<jupyter_output>Enter the user id: 6


Recommonded Top 10 movies for userId 6 are listed below

1 --> Denise Calls Up (1995)
2 --> Babes in Toyland (1934)
3 --> On the Beach (1959)
4 --> Bossa Nova (2000)
5 --> Man with the Golden Arm, The (1955)
6 --> Slumber Party Massacre II (1987)
7 --> Slumber Party Massacre III (1990)
8 --> Sorority House Massacre (1986)
9 --> Sorority House Massacre II (1990)
10 --> Two Family House (2000)<jupyter_text>==========================================================================  Display the MovieID, IMDB ID, Average User Rating (excluding predictions), genres and tag of all the movies found in Step 3 as a data frame.  Finding the mean rating for all the movies<jupyter_code>avg_rating = {}

for i in range(1,193609):
    if (i==33)|(i==35)|(i==37):         # In these movieId the rating is missing so we will no considered it and we will pass
        pass
    else:
        avg_rating[i] = round((ratings[ratings.movieId == i]['rating'].mean()),2) # finding avg rating

#print(avg_rating)<jupyter_output><empty_output><jupyter_text>Merging all the datasets<jupyter_code>md = pd.merge(ratings[["movieId","rating"]],tag[["movieId","tag"]],how="left",on="movieId")
md2 = pd.merge(md,links[["movieId","imdbId"]],how="left",on="movieId")
md3 = pd.merge(md2,movies[["movieId","genres"]],how="left",on="movieId")
md3.head()<jupyter_output><empty_output><jupyter_text>Printing MovieID, IMDB ID,Average User Rating,genres and tag for all the movies of above userid<jupyter_code>ls = []

for i in range(10):
    ls.append(high[i][0])
md4 = md3[md3["movieId"].isin(ls)]

ls
md4<jupyter_output><empty_output><jupyter_text>Now dropping rating column and as we need the avg_rating of movies in our dataframe<jupyter_code>#md4 = md4.drop("rating",axis=1) #once you drop this column do not run it just comment it
#md4
id_rating_pair = [(x,y) for x,y in avg_rating.items() if x in ls]

new_rating = []
for i in range(len(id_rating_pair)):
    new_rating.append(id_rating_pair[i][1])
new_rating<jupyter_output><empty_output><jupyter_text>Now we are Sorting movieId in ascending order and adding a new column avg_rating<jupyter_code>md5 = md4.sort_values(by="movieId")
md5["avg_rating"] = new_rating
md5<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12715456343599887, 'fraction_numerical': 0.032965997927851556, 'mean_word_length': 5.439053972104306, 'pattern_counts': {'":': 0, '<': 89, '<?xml version=': 0, '>': 101, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '188831', 'license': 'BSD-4-Clause-UC', 'n_tokens_mistral': 3834, 'n_tokens_neox': 3493, 'n_words': 1009, 'path': 'Movie_Recommendation.ipynb', 'repo_name': 'shashankshanu102/Recommender_System', 'size': '10623'}