<jupyter_start><jupyter_text>We are going to use imdb dataset to create content based recommended system.Datasets = https://drive.google.com/file/d/1bPm6emwAUgCwGs_jX6FGAUTPUCDcvApR/view?usp=sharing,%20https://drive.google.com/file/d/1urB8DUCOEMAbMyo1sMBTkcKm7JquZ2dE/view?usp=sharing<jupyter_code>import pandas as pd
import numpy as np 
credits = pd.read_csv("/content/tmdb_5000_credits.csv")
movies = pd.read_csv("/content/tmdb_5000_movies.csv")
(movies.info())
(movies.head())
credits.info()
credits.head()
print('credits :',credits.shape)
print('movies :',movies.shape)<jupyter_output>credits : (4803, 4)
movies : (4803, 20)<jupyter_text>Now we will merge both the datasets on the basis of movie id<jupyter_code>credits_renamed = credits.rename(index=str,columns={"movie_id":"id"})
credits_renamed = credits.drop(columns=['title'])
movies_credits_merge= movies.merge(credits_rename, on='id')
(movies_credits_merge.head())
movies_cleaned =movies_credits_merge.drop(columns=['homepage','status','title_x','title_y','production_countries'])
movies_cleaned.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 4803 entries, 0 to 4802
Data columns (total 18 columns):
 #   Column                Non-Null Count  Dtype  
---  ------                --------------  -----  
 0   budget                4803 non-null   int64  
 1   genres                4803 non-null   object 
 2   id                    4803 non-null   int64  
 3   keywords              4803 non-null   object 
 4   original_language     4803 non-null   object 
 5   original_title        4803 non-null   object 
 6   overview              4800 non-null   object 
 7   popularity            4803 non-null   float64
 8   production_companies  4803 non-null   object 
 9   release_date          4802 non-null   object 
 10  revenue               4803 non-null   int64  
 11  runtime               4801 non-null   float64
 12  spoken_languages      4803 non-null   object 
 13  tagline               3959 non-null   object 
 14  vote_average          4803 non-null   float64
 15  vote_count           [...]<jupyter_text>Now we are going to use movie summaries to create a vector for each movie in which we will use the words in our summary. Then we will compare vectors of all movies and recommend those which will have similar values.We will use tfidf from sklearn to implement this.<jupyter_code>from sklearn.feature_extraction.text import TfidfVectorizer
tfv = TfidfVectorizer(min_df = 3,
                      max_features=None,
                      strip_accents='unicode',
                      analyzer='word',
                      token_pattern= 'r\w{1,}', # \w{1,} = alphanum with minimum 1 letter
                      ngram_range=(1,3), # will create combination  sets of 1 to 3 words
                      stop_words='english')

# we fill fit the tfidf on summary text
print(movies_cleaned['overview'])
tfv_matrix = tfv.fit_transform(movies_cleaned['overview'].values.astype('U')) #astype converts it to unicode
print(tfv_matrix, ' \n', tfv_matrix.shape)


# so now we have vectors 

from sklearn.metrics.pairwise import sigmoid_kernel

'''sigmoid kernel basically calculates hyperbolic tangent 
betw two vectors,i.e,distance between 2 vectors
This will show us how  similar two movie summaries are based 
on the distances betw vector values'''

sigma = sigmoid_kernel(tfv_matrix,tfv_matrix)
print(sig[0])
# Now we will map these values with their titles 

# we have created a Series which has title:index which is hashed
indices = pd.Series(movies_cleaned.index, index = movies_cleaned['original_title']).drop_duplicates()

print(indices)
print(indices['Avatar'])
print(sig[4802])
print(list(enumerate(sigma[indices['Avatar']]))) # this shows us vector of avatar compared with all the movies
# now we have sorted the list based on vector values closest to our given movie
print(sorted(list(enumerate(sigma[indices['Avatar']])),key=lambda x: x[1], reverse=True))

# now we will give create a function to give 10 recommendations based on movie title

def give_recommendations(title):
    #get index
    idx= indices[title]


    #Get similarity score
    sigma_scores = list(enumerate(sigma[idx]))

    #sort the movies according to scores
    sigma_scores = sorted(sigma_scores, key= lambda x: x[1], reverse=True)

    #get 10 most similar movies
    sigma_scores =sigma_scores[1:11]
   # print(sigma_scores)

    #get Movie indices
    movie_indices = [ i[0] for i in sigma_scores]
    #print(movie_indices)

    #print 10 similar movies
    print('Movies similar to ',title," are : \n")
    return movies_cleaned['original_title'].iloc[movie_indices]


print(give_recommendations('Spectre'),'\n')
print(give_recommendations("The Dark Knight Rises"))

# this was a very basic attempt at doing this , I will be fine tuning this...<jupyter_output>Movies similar to  Spectre  are : 

3162                    Thunderball
1281                        Hackers
3351    The Man with the Golden Gun
1967                The Fisher King
367                 The Interpreter
4402            In Her Line of Fire
3285                       Restless
753                    The Sentinel
1200           The Living Daylights
2933                       F.I.S.T.
Name: original_title, dtype: object 

Movies similar to  The Dark Knight Rises  are : 

299                            Batman Forever
2507                                Slow Burn
65                            The Dark Knight
1181                                      JFK
879                       Law Abiding Citizen
1202                             Legal Eagles
3595    Halloween: The Curse of <NAME>
307                         The Expendables 3
2193                     Secret in Their Eyes
42                                Toy Story 3
Name: original_title, dtype: object
{'fraction_non_alphanumeric': 0.08244406196213425, 'fraction_numerical': 0.04044750430292599, 'mean_word_length': 2.5848241826033314, 'pattern_counts': {'":': 1, '<': 12, '<?xml version=': 0, '>': 12, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '113167', 'license': 'MIT', 'n_tokens_mistral': 1914, 'n_tokens_neox': 1687, 'n_words': 530, 'path': 'Movie_recommendation_system.ipynb', 'repo_name': 'AdityaAgarkar1/Movie-Recommendation-', 'size': '5817'}