<jupyter_start><jupyter_text>Matrix FactorizationIn the last notebook you gained your first opportunity in working with the `turicreate` library.  In this notebook, you will continue working with this library to make recommendations.  However, you will use matrix factorization techniques beyond the `popularity` and `item similarity` recommendations you made in the previous notebook. Import DataStart below by reading in the data and libraries needed for this task.<jupyter_code>import json
import numpy as np
import pandas as pd
import turicreate as tc

ratings_dat = pd.read_csv('../../../data/ratings.dat', sep='::', engine='python', \
                          header=None, names=['user_id', 'movie_id','rating','time'])

ratings_dat2 = ratings_dat.copy(deep=True)
ratings_dat2.columns = ['user_id', 'item_id', 'rating', 'time']
ratings_sframe = tc.SFrame(ratings_dat2[['user_id', 'item_id', 'rating']])

ratings_sframe.head()<jupyter_output><empty_output><jupyter_text>Before we dive into building a recommendation engine using matrix factorization, let's take some time to make sure you understand the ideas associated with this technique.**Question 1:** When performing matrix factorization, one of the created matrices from this technique holds measurements about how a user feels... **[complete this sentence]**.<jupyter_code>import data_solution_part3 as sp

a = "towards other users"
b = "towards each item"
c = "as counts of how often they interacted with an item"
d = "as latent factors"

your_answer = d

sp.answer_one(your_answer)<jupyter_output>That's right! Matrix factorization provides matrices that provide a user's latent (not explicit) feelings towards different elements of the items.<jupyter_text>**Question 2:** When performing matrix factorization, a second matrix contains... **[complete this sentence]**.<jupyter_code>a = "latent factors related to items"
b = "how users relate to one another"
c = "how items relate to one another"
d = "counts of how often users interacted with an item"

your_answer = a

sp.answer_two(your_answer)<jupyter_output>That's right! Matrix factorization provides one matrix related to the latent factors associated with a user, and a second matrix related to the latent factors associated with the item.<jupyter_text>In certain matrix factorization methods ([FunkSVD](https://sifter.org/~simon/journal/20061211.html)), which is the most popular method used with recommendation systems, you may only retrieve the two matrices considered in the previous two questions.  In other matrix factorization techniques (Singular Value Decomposition), which are generally not possible given the missing values associated with recommendation systems, you may also obtain a third matrix.  [The series does a great job of walking you through the why and how in the posts through July 30 of all of the details if you are interested](http://nicolas-hug.com/tagssvd).**Question 3:** This third matrix contains... **[complete this sentence]**.<jupyter_code>a = "mostly 0's"
b = "mostly 1's"
c = "mostly 2's"
d = "other"

your_answer = a

sp.answer_three(your_answer)<jupyter_output>That's right! The third (middle) matrix is a matrix with 0's everywhere except along the diagonal.  The values along the diagonal give a measurement of how much each latent factor matters towards creating the original matrix.<jupyter_text>Now you will get your first opportunity at applying a factorization technique to making recommendations. The truth is that `turicreate` hides a lot of the details related to this technique.  In fact the process for using matrix factorization in `turicreate` is essentially the same as the techniques you saw in the previous notebook.  However, matrix factorization methods usually significantly outperform the methods you saw in the previous section in terms of their ability to make recommendations that will leave users delighted (or at least to click more).**Question 4:** Use the skeleton of the process below and the `ratings_sframe` to complete your recommendations using matrix factorization.  If you need guidance, use the link [here](https://apple.github.io/turicreate/docs/userguide/recommender/using-trained-models.html).<jupyter_code># use create to fit recommendation
model_factorization = tc.factorization_recommender.create(ratings_sframe, target='rating')

# create an sframe of user_ids to make recs for user 1
users_test = tc.SFrame({'user_id': [1]})

# use recommend to make recommendations for the set of users
factorization_results = model_factorization.recommend(users_test)

# the top 5 recommendations for user_id 1 are:
top_5_recs = factorization_results['item_id'][:5]

sp.answer_four(set(top_5_recs))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07713498622589532, 'fraction_numerical': 0.006145369781733418, 'mean_word_length': 5.601398601398602, 'pattern_counts': {'":': 0, '<': 18, '<?xml version=': 0, '>': 18, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '180324', 'license': 'MIT', 'n_tokens_mistral': 1285, 'n_tokens_neox': 1216, 'n_words': 561, 'path': 'Part 3 - Recommendation Systems/notebooks/solutions/matrix_factorization-Solution.ipynb', 'repo_name': 'tongni1975/rec-workshop', 'size': '4719'}