<jupyter_start><jupyter_text>Assignment: 1. Perform some movie recommendations and analysis for user 2:- How many movies has this user watched?- Plot a bar chart of their movie ratings. The bar chart should be the counts of the number of unique ratings.    - Hint: the `sort_index()` function from pandas might be helpful to make the bar plot look nicer.- What are some of user 2's top movies?    - Hint: to get the actual movie titles, you can use pandas [merge](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.merge.html) function, although using the movie IDs is OK too.- Find the most similar user in the movielens dataset to user 2 using at least 2 distance metrics. Be sure to use cosine distance as one of your choices.- Recommend a few movies for user 2 using similarity metrics.- Do the recommendations from this method make sense?- Write a short analysis of the results, and justify which similarity metric(s) you used.Optional challenges:- Perform other analyses (e.g. EDA, visualizations) of the movies watched from this dataset, or from a bigger part of the dataset for the movielens dataset: https://grouplens.org/datasets/movielens/- Add yourself as a user in the data with ratings for movies you've watched, and find recommendations for next movies to watch. - Use a more advanced collaborative or content-based recommender to make recommendations (e.g. using the surprise package in Python)    - Try making predictions for user 2. How do they compare with our basic model?    - Add your own movie ratings, or use another recommender dataset and add your own preferences, then get recommendations for yourself<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
ratings = pd.read_csv('ratings.csv')
movies = pd.read_csv('movies.csv')
movies.head()
ratings.head()
df = ratings.merge(movies, how='inner', on='movieId')
df<jupyter_output><empty_output><jupyter_text>How many movies has this user watched?<jupyter_code>user2_unique = df[df['userId'] == 2.0].nunique()
movies_seen_by_user2 = user2_unique['movieId']
f'User 2 has rated {movies_seen_by_user2} movies'<jupyter_output><empty_output><jupyter_text>Plot a bar chart of their movie ratings. The bar chart should be the counts of the number of unique ratings.<jupyter_code>user2_ratings = df[(df['userId'] == 2.0) & (df['rating'] >= 0.0)]
frequency = user2_ratings['rating'].value_counts()
print('rating', 'frequency')
print(frequency)
ax = frequency.plot(kind='bar',
               title="User 2 Unique Ratings",
              figsize=(5,5))
ax.set(xlabel='Rating',
       ylabel='Frequency')<jupyter_output><empty_output><jupyter_text>What are some of user 2's top movies?<jupyter_code>user2_movies = df[df['userId'] == 2.0]
top_user2_movies = user2_movies.sort_values(by='rating', ascending=False)
top_user2_movies[top_user2_movies['rating'] == 5.0]<jupyter_output><empty_output><jupyter_text>Find the most similar user in the movielens dataset to user 2 using at least 2 distance metrics. Be sure to use cosine distance as one of your choices.<jupyter_code>wide = df.pivot(index='userId', columns='movieId', values='rating')
wide.head()
cor = wide.T.corr()
cor.head()
cor.loc[2].sort_values(ascending=False)
wide.fillna(-1, inplace=True)
def euclidean_distance(vector1, vector2):
    return np.sqrt(np.sum((vector1 - vector2) ** 2))

def euclidean_distance(vector1, vector2):
    return np.linalg.norm(vector1 - vector2)
    
from scipy.spatial.distance import euclidean
from scipy.spatial.distance import pdist, squareform
euclidean_distances = squareform(pdist(wide, metric=euclidean))
euclidean_df = pd.DataFrame(data=euclidean_distances, columns=wide.index, index=wide.index)
euclidean_df.loc[2].sort_values()
cosine_distances = squareform(pdist(wide, metric='cosine'))
cosine_df = pd.DataFrame(cosine_distances, columns=wide.index, index=wide.index)
cosine_df.loc[2].sort_values()
euclidean = euclidean_df.loc[2].sort_values().loc[422]
cosine = cosine_df.loc[2].sort_values().loc[422]
print('User 422s movie preferences are most similar to User 2s:')
print('Euclidean Distance: %.3f' %euclidean)
print('Cosine Distance: %.3f' %cosine)<jupyter_output>User 422s movie preferences are most similar to User 2s:
Euclidean Distance: 50.222
Cosine Distance: 0.120<jupyter_text>Recommend a few movies for user 2 using similarity metrics.<jupyter_code>wide = df.pivot(index='userId', columns='movieId', values='rating')
rated_5_by_422_not_watched_by_2 = (wide.loc[422] == 5) & (wide.loc[2].isna())
user2_notseen = wide.loc[2][rated_5_by_422_not_watched_by_2]
user422_5 = wide.loc[422][rated_5_by_422_not_watched_by_2]
print(wide.loc[2][rated_5_by_422_not_watched_by_2])
print(wide.loc[422][rated_5_by_422_not_watched_by_2])
wide.loc[422].sort_values(ascending=False)
R1 = df[(df['userId'] == 422) & (df['movieId'] == 1213)]
R2 = df[(df['userId'] == 422) & (df['movieId'] == 922)]
R3 = df[(df['userId'] == 422) & (df['movieId'] == 2728)]
R4 = df[(df['userId'] == 422) & (df['movieId'] == 1193)]
R5 = df[(df['userId'] == 422) & (df['movieId'] == 1387)]
print('Top 5 Movie Recommendations for User 2 ')
print()
print('Movie 1:  ')
print(R1['title'])
print(R1['genres'])
print()
print('Movie 2:  ')
print(R2['title'])
print(R2['genres'])
print()
print('Movie 3:  ')
print(R3['title'])
print(R3['genres'])
print()
print('Movie 4:  ')
print(R4['title'])
print(R4['genres'])
print()
print('Movie 5:  ')
print(R5['title'])
print(R5['genres'])<jupyter_output>Top 5 Movie Recommendations for User 2 

Movie 1:  
7846    Goodfellas (1990)
Name: title, dtype: object
7846    Crime|Drama
Name: genres, dtype: object

Movie 2:  
39955    Sunset Blvd. (a.k.a. Sunset Boulevard) (1950)
Name: title, dtype: object
39955    Drama|Film-Noir|Romance
Name: genres, dtype: object

Movie 3:  
83162    Spartacus (1960)
Name: title, dtype: object
83162    Action|Drama|Romance|War
Name: genres, dtype: object

Movie 4:  
48949    One Flew Over the Cuckoo's Nest (1975)
Name: title, dtype: object
48949    Drama
Name: genres, dtype: object

Movie 5:  
71201    Jaws (1975)
Name: title, dtype: object
71201    Action|Horror
Name: genres, dtype: object
{'fraction_non_alphanumeric': 0.12550738756291605, 'fraction_numerical': 0.04107809709368404, 'mean_word_length': 5.681127982646421, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 24, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '307332', 'license': 'MIT', 'n_tokens_mistral': 2275, 'n_tokens_neox': 2046, 'n_words': 621, 'path': 'assignment/week6/Week_6_assignment_starter.ipynb', 'repo_name': 'kscully-dotcom/msds600', 'size': '6159'}