<jupyter_start><jupyter_text>Machine Learning Foundation Specialization  University of Washington - Seattle  Week 5 Quiz2 Recommending Songs<jupyter_code>## Created by <NAME>
## Created on 05/02/2020 23:25
## <EMAIL>
## Mayo Clinic College of Medicine and Sciences<jupyter_output><empty_output><jupyter_text>**Instructions:** 1. **Counting unique users:** The method .unique() can be used to select the unique elements in a column of data. In this question, you will compute the number of unique users who have listened to songs by various artists. For example, to find out the number of unique users who listened to songs by 'Kanye West', all you need to do is select the rows of the song data where the artist is 'Kanye West', and then count the number of unique entries in the ‘user_id’ column. Compute the number of unique users for each of these artists: 'Kanye West', 'Foo Fighters', 'Taylor Swift' and 'Lady GaGa'. **Save these results to answer the quiz at the end**.<jupyter_code>import turicreate
song_data = turicreate.SFrame('song_data.sframe')
song_data.head()
turicreate.visualization.set_target('auto')
song_data['artist'].show()
users = song_data['user_id'].unique()
Kanye_West = len(song_data[song_data['artist']=='Kanye West']['user_id'].unique())
Kanye_West
Foo_Fighters = len(song_data[song_data['artist']=='Foo Fighters']['user_id'].unique())
Foo_Fighters
Taylor_Swift = len(song_data[song_data['artist']=='Taylor Swift']['user_id'].unique())
Taylor_Swift
Lady_GaGa = len(song_data[song_data['artist']=='Lady GaGa']['user_id'].unique())
Lady_GaGa<jupyter_output><empty_output><jupyter_text>2. **Using groupby-aggregate to find the most popular and least popular artist:** each row of song_data contains the number of times a user listened to particular song by a particular artist. If we would like to know how many times any song by 'Kanye West' was listened to, we need to select all the rows where ‘artist’=='Kanye West' and sum the ‘listen_count’ column. If we would like to find the most popular artist, we would need to follow this procedure for each artist, which would be very slow. Instead, you will learn about a very important method:<jupyter_code># .groupby()<jupyter_output><empty_output><jupyter_text>You can read the [documentation about groupby here.](https://turi.com/products/create/docs/generated/graphlab.SFrame.groupby.htmlgraphlab.SFrame.groupby) The .groupby method computes an aggregate (in our case, the sum of the ‘listen_count’) for each distinct value in a column (in our case, the ‘artist’ column).  Follow these steps to find the most popular artist in the dataset: * The .groupby method has two important parameters: i. key_columns, which takes the column we want to group, in our case, ‘artist’ ii. operations, where we define the aggregation operation we using, in our case, we want to sum over the ‘listen_count’. * With this in mind, the following command will compute the sum listen_count for each artist and return an SFrame with the results:<jupyter_code>popular_artist = song_data.groupby('artist', operations={'total_count': turicreate.aggregate.SUM('listen_count')})
popular_artist.head()<jupyter_output><empty_output><jupyter_text>the total number of listens for each artist will be stored in ‘total_count’. * Sort the resulting SFrame according to the ‘total_count’, and find the artist with the most popular and least popular artist in the dataset. **Save these results to answer the quiz at the end**.<jupyter_code>most_popular_artist = popular_artist.sort('total_count',ascending = False)
most_popular_artist
most_popular_artist['artist'][0]
least_popular_artist = popular_artist.sort('total_count',ascending = True)
least_popular_artist.head()
least_popular_artist['artist'][0]<jupyter_output><empty_output><jupyter_text>3. **[OPTIONAL] Using groupby-aggregate to find the most recommended songs:** Now that we learned how to use .groupby() to compute aggregates for each value in a column, let’s use to find the song that is most recommended by the personalized_model model we learned in the Jupyter notebook above. Follow these steps to find the most recommended song: * Split the data into 80% training, 20% testing, using seed=0, as was done in the Jupyter notebook above.<jupyter_code>train_data,test_data = song_data.random_split(.8,seed=0)<jupyter_output><empty_output><jupyter_text>[Optional] Train an popularity_recommender, as done in the Jupyter notebook, using the training data.<jupyter_code>popularity_model = turicreate.popularity_recommender.create(train_data,
                                                           user_id = 'user_id',
                                                           item_id = 'song')<jupyter_output><empty_output><jupyter_text>Predictions based on the Popularity Recommender System<jupyter_code>popularity_model.recommend(users=[users[0]])
popularity_model.recommend(users=[users[1]])<jupyter_output><empty_output><jupyter_text>* Train an item_similarity_recommender, as done in the Jupyter notebook, using the training data.<jupyter_code>personalized_model = turicreate.item_similarity_recommender.create(train_data,
                                                                  user_id = 'user_id',
                                                                  item_id = 'song')<jupyter_output><empty_output><jupyter_text>* Next, we are going to make recommendations for the users in the test data, but there are over 200,000 users (58,628 unique users) in the test set. Computing recommendations for these many users can be slow in some computers. Thus, we will use only the first 10,000 users only in this question. Using this command to select this subset of users:<jupyter_code>subset_test_users = test_data['user_id'].unique()[0:10000]<jupyter_output><empty_output><jupyter_text>* Let’s compute one recommended song for each of these test users. Use this command to compute these recommendations:<jupyter_code>personalized_model.recommend(subset_test_users,k=1)<jupyter_output><empty_output><jupyter_text>* Finally, we can use .groupby() to find the most recommended song! :) When we used .groupby() in the previous question, we summed up the total ‘listen_count’ for each artist, by setting the parameter SUM in the aggregator:<jupyter_code># operations={'total_count': turicreate
# .aggregate.SUM('listen_count')}
popular_artist = song_data.groupby('artist', operations={'total_count': turicreate.aggregate.SUM('listen_count')})<jupyter_output><empty_output><jupyter_text>For this question, we simply want to count how often each song is recommended, so we will use the COUNT aggregator instead of SUM, and store the results in a column we will call ‘count’ by using:<jupyter_code>updated_popular_artist = song_data.groupby('artist',operations={'count': turicreate.aggregate.COUNT()})
updated_popular_artist.head()
updated_popular_artist.sort('count',ascending=True)
updated_popular_artist.sort('count',ascending=False)<jupyter_output><empty_output><jupyter_text>And, since we want to use the song titles as the key to the aggregator instead of of the ‘artist’, we use:<jupyter_code># key_columns='song'
popular_song = song_data.groupby('song',operations={'count': turicreate.aggregate.COUNT()})
popular_song.head()<jupyter_output><empty_output><jupyter_text>* By sorting the results, you will find out the most recommended song to the first 10,000 users in the test data! Due to randomness in train-test split, the most recommended song may come out differently for different people. This is why we chose not to assign a quiz question for this section.<jupyter_code>popular_song.sort('count',ascending=False)
popular_song.sort('count',ascending=True)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09007257646448938, 'fraction_numerical': 0.007257646448937273, 'mean_word_length': 5.545377438507209, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '101706', 'license': 'MIT', 'n_tokens_mistral': 2325, 'n_tokens_neox': 2187, 'n_words': 794, 'path': 'Week_5/Quiz_2/W5_Q2.ipynb', 'repo_name': 'quincy-125/ML_Foundation_UW-Seattle', 'size': '7734'}