<jupyter_start><jupyter_text>Model based Collaborative Filtering - this notebook did not complete due to memory limitations   https://towardsdatascience.com/how-to-build-a-model-based-recommendation-system-using-python-surprise-2df3b77ab3e5 > [How to Build a Memory-Based RecSys using Python Surprise](https://towardsdatascience.com/how-to-build-a-model-based-recommendation-system-using-python-surprise-2df3b77ab3e5)> - Model-based models (admittedly, a weird name) use some sort of machine learning algorithm to estimate the ratings. A typical example is singular value decomposition of the user-item ratings matrix.  Binary ratings<jupyter_code>from surprise import Dataset, Reader
from time import time
from surprise.model_selection import train_test_split
from surprise import accuracy
from surprise import SVD
import pandas as pd
import numpy as np
file_path = '../data-processed/scale-down/pid-track-binary-rating-train-data-scaled.csv'
reader = Reader(
     line_format='user item rating', sep=',', rating_scale = (0,1), skip_lines=1
     )
data = Dataset.load_from_file(file_path, reader=reader)
trainsetfull = data.build_full_trainset()
print('Number of users: ', trainsetfull.n_users, '\n')
print('Number of items: ', trainsetfull.n_items, '\n')
trainset, testset = train_test_split(data, test_size=0.2)
print('Number of users: ', trainset.n_users, '\n')
print('Number of items: ', trainset.n_items, '\n')
trainset_iids = list(trainset.all_items())
iid_converter = lambda x: trainset.to_raw_iid(x)
trainset_raw_iids = list(map(iid_converter, trainset_iids))
trainsetfull_iids = list(trainsetfull.all_items())
iid_converter = lambda x: trainsetfull.to_raw_iid(x)
trainsetfull_raw_iids = list(map(iid_converter, trainsetfull_iids))
SVD_model = SVD(n_factors=500)
SVD_model.fit(trainset)
predictions = SVD_model.test(testset)
accuracy.rmse(predictions)
train_set=pd.read_csv(file_path)
evaluation_set = pd.read_csv('../data-processed/full-data/evaluation-pids-ground-truth.csv')
(trainset_iids,trainset_raw_iids)
evaluation_set['pred_svd'] = evaluation_set.apply(lambda x: SVD_model.predict(uid = int(x.pid), iid = x.track_uri)[4], axis=1)
evaluation_set['pred_rating'] = evaluation_set.apply(lambda x: SVD_model.predict(uid = int(x.pid), iid = x.track_uri)[3], axis=1)
evaluation_set.head(50)
tracks = list(train_set['track_uri'].unique())
tracks[:10]<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12133891213389121, 'fraction_numerical': 0.011715481171548118, 'mean_word_length': 6.813725490196078, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '275422', 'license': 'MIT', 'n_tokens_mistral': 842, 'n_tokens_neox': 820, 'n_words': 170, 'path': 'modeling-notebooks/.ipynb_checkpoints/CF01_Model_Surprise_scaled_down-checkpoint.ipynb', 'repo_name': 'IrynaHorova/APC-Spotify-Million-Playlists-Ironhack-Project', 'size': '2390'}