<jupyter_start><jupyter_text>Sistemas de recomendación Carga de datos de Movie Lens<jupyter_code>import pandas as pd
df = pd.read_csv("../datasets/ml-100k/u.data.csv", sep="\t", header=None)
type(df)
df.head()
df.shape
df.columns = ["UserID", "ItemID", "Rating", "TimeStamp"]
df.head()<jupyter_output><empty_output><jupyter_text>Análisis exploratorio de los ítems<jupyter_code>import matplotlib.pyplot as plt
plt.hist(df.Rating)
plt.hist(df.TimeStamp)
df.groupby(["Rating"])["UserID"].count()
plt.hist(df.groupby(["ItemID"])["ItemID"].count())<jupyter_output><empty_output><jupyter_text>Representación en forma matricial<jupyter_code>import numpy as np
n_users = df.UserID.unique().shape[0]
n_users
n_items = df.ItemID.unique().shape[0]
n_items
ratings = np.zeros((n_users, n_items))
for row in df.itertuples():
    ratings[row[1]-1, row[2]-1] = row[3]
type(ratings)
ratings.shape
ratings
sparsity = float(len(ratings.nonzero()[0]))
sparsity /= (ratings.shape[0]*ratings.shape[1])
sparsity *= 100
print("Coeficiente de sparseidad: {:4.2f}%".format(sparsity))<jupyter_output>Coeficiente de sparseidad: 6.30%<jupyter_text>Crear conjuntos de entrenamiento y validación<jupyter_code>from sklearn.model_selection import train_test_split
ratings_train, ratings_test = train_test_split(ratings, test_size = 0.3, random_state=42)
ratings_train.shape
ratings_test.shape<jupyter_output><empty_output><jupyter_text>Filtro colaborativo basado en Usuarios* Matriz de similaridad entre los usuarios (distancia del coseno).* Predecir la valoración desconocida de un ítem *i* para un usuario activo *u* basandonos en la suma ponderada de todas las valoraciones del resto de usuarios para dicho ítem.* Recomendaremos los nuevos ítems a los usuarios según lo establecido en los pasos anteriores.<jupyter_code>import numpy as np
import sklearn
sim_matrix = 1 - sklearn.metrics.pairwise.cosine_distances(ratings_train)
type(sim_matrix)
sim_matrix.shape
sim_matrix
users_predictions = sim_matrix.dot(ratings_train) / np.array([np.abs(sim_matrix).sum(axis=1)]).T
users_predictions
from sklearn.metrics import mean_squared_error
def get_mse(preds, actuals):
    if preds.shape[0] != actuals.shape[0]:
        actuals = actuals.T
    preds = preds[actuals.nonzero()].flatten()
    actuals = actuals[actuals.nonzero()].flatten()
    return mean_squared_error(preds, actuals)
get_mse(users_predictions, ratings_train)
sim_matrix = 1 - sklearn.metrics.pairwise.cosine_distances(ratings_test)
users_predictions = sim_matrix.dot(ratings_test) / np.array([np.abs(sim_matrix).sum(axis=1)]).T
get_mse(users_predictions, ratings_test)<jupyter_output><empty_output><jupyter_text>Filtro colaborativo basado en los KNN<jupyter_code>from sklearn.neighbors import NearestNeighbors
k = 5
neighbors = NearestNeighbors(k, 'cosine')
neighbors.fit(ratings_train)
top_k_distances, top_k_users = neighbors.kneighbors(ratings_train, return_distance=True)
top_k_distances.shape
top_k_distances[0]
top_k_users.shape
top_k_users[0]
users_predicts_k = np.zeros(ratings_train.shape)
for i in range(ratings_train.shape[0]):# para cada usuario del conjunto de entrenamiento
    users_predicts_k[i,:] = top_k_distances[i].T.dot(ratings_train[top_k_users][i]) / np.array([np.abs(top_k_distances[i].T).sum(axis=0)]).T
users_predicts_k.shape
users_predicts_k
get_mse(users_predicts_k, ratings_train)
users_predicts_k = np.zeros(ratings_test.shape)
for i in range(ratings_test.shape[0]):# para cada usuario del conjunto de test
    users_predicts_k[i,:] = top_k_distances[i].T.dot(ratings_test[top_k_users][i]) / np.array([np.abs(top_k_distances[i].T).sum(axis=0)]).T
get_mse(users_predicts_k, ratings_test)
ratings_test<jupyter_output><empty_output><jupyter_text>Filtro colaborativo basado en Items<jupyter_code>n_movies = ratings_train.shape[1]
n_movies
neighbors = NearestNeighbors(n_movies, 'cosine')
neighbors.fit(ratings_train.T)
top_k_distances, top_k_items = neighbors.kneighbors(ratings_train.T, return_distance=True)
top_k_distances.shape
top_k_distances
top_k_items.shape
top_k_items
item_preds = ratings_train.dot(top_k_distances) / np.array([np.abs(top_k_distances).sum(axis=1)])
item_preds.shape
item_preds
get_mse(item_preds, ratings_train)
get_mse(item_preds, ratings_test)<jupyter_output><empty_output><jupyter_text>Filtrado colaborativo basado en KNN<jupyter_code>k = 30
neighbors = NearestNeighbors(k, 'cosine')
neighbors.fit(ratings_train.T)
top_k_distances, top_k_items = neighbors.kneighbors(ratings_train.T, return_distance=True)
top_k_distances.shape
top_k_items[0]
top_k_distances[0]
preds = np.zeros(ratings_train.T.shape)
for i in range(ratings_train.T.shape[0]):
    if(i%50==0):
        print("iter "+str(i))
    den = 1
    if (np.abs(top_k_distances[i]).sum(axis=0)>0):
        den = np.abs(top_k_distances[i]).sum(axis=0)
    preds[i, :] = top_k_distances[i].dot(ratings_train.T[top_k_items][i])/np.array([den]).T
get_mse(preds, ratings_train)
get_mse(preds, ratings_test)
preds.shape
ratings_train.shape
ratings_test.shape<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12373184802068829, 'fraction_numerical': 0.01034414163517008, 'mean_word_length': 7.79020979020979, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '220452', 'license': 'MIT', 'n_tokens_mistral': 2045, 'n_tokens_neox': 1987, 'n_words': 318, 'path': 'notebooks/T9 - 3 - Sistemas de recomendación.ipynb', 'repo_name': '1214345/python-ml-course', 'size': '5027'}