<jupyter_start><jupyter_text>I Don't Know - Where Do You Want to Eat? 🍽️🤷‍♀️🤷‍♀️🤷‍♀️🥢  Section 1) Introduction 📗  It's a tough day at the office, and lunch time is here.  Have you ever had the following exchange with your co-workers and/ or friends? 🤔: Where do you want to eat? 🤨: I don't know.  Where do you want to eat? 🙄: I don't know.  Where do you want to eat? And on and on and on 🤯.  Before you know it, lunch time is over, and it's time to get back to the gridstone.  You've spent most (if not all) of your lunch time simply trying to figure out where to eat.  We live in a world where we are faced with unprecidented choice in our options for entertainment, food, and habitation.  Being spolied for choice is a double-edged sword though.  In the face of so many options, people are often not happy with the choices that they make experiencing either buyer's remorse (fear of missing out aka FOMO) or being completely overwhelmed leading to analysis paralysis (AP).  My project (I Don't Know - Where Do You Want to Eat) aims to help lunch buddies solve the paradox of choice by implmeneting a hybrid recommender system (both content and collaborative filtering) to help find restaurants that meet the group's dining preferences and offering them 3 options that are close to their central location (the lunch buddies don't have to be at the same location).  The locations of the participants will be used to create a centroid (central location) between them that will be used in conjunction with queries the FourSquare API to return candidate restaurants fitting the top criteria of the diners.  I will then use the matrix factorization techique to create a sparse matrix that I will then feed to a LightFM model to fill in any blanks in the utilization matrix so that the top 3 recommendations can be returned and visualized.  Section 2) Data 💾  I will be using the following data sources to solve this problem: - FourSquare API: To make calls to retrieve venues using the `venues\search` and `venues\VENUEID` endpoints.  This will be the basis for the item matrix that will help generate the utilization matrix as well as providing crucial information for the visualization of the final venue selections and their attributes (ratings, addresses, categories, etc.).  I will also make a call to the `venues\categories`  endpoint to query all food venue category ids.  The users will also use this in order to construct their corresponding profiles. - My dear friends and colleagues: My friends and colleauges have generously agreed to create user profiles along with their respective lunch time locations and dining preferences.  I will use this information along with the FourSquare Categories to create the user matrix.<jupyter_code># First things first
! conda install -c conda-forge folium --yes
! conda install -c conda-forge pandas-profiling --yes
! conda install -c conda-forge wordcloud --yes
! pip install lightfm
! pip install geocoder
# Let's get into it!

# standard weapons of choice
import pandas as pd
from pandas.io.json import json_normalize
import numpy as np
import scipy
from scipy.sparse import csr_matrix
from sklearn.model_selection import RandomizedSearchCV
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.cluster import KMeans
from sklearn.metrics.pairwise import cosine_similarity
import pandas_profiling

# for geolocation assistance
import geocoder

# for requesting
import requests

# for exporting files to IBM Cloud
import io

# all the viz
import folium
import matplotlib.pyplot as plt
import seaborn as sns
from wordcloud import WordCloud
from PIL import Image

# for the hybrid recommender system
from lightfm import LightFM
from lightfm.evaluation import precision_at_k

%matplotlib inline
# The code was removed by Watson Studio for sharing.
# Creating the url for the categories API call
category_url = f'https://api.foursquare.com/v2/venues/categories?client_id={CLIENT_ID}&client_secret={CLIENT_SECRET}&v={VERSION}'

# Creating the nub of the Food category dataframe with the parent cat name and id
json_response = requests.get(category_url).json()
food_category = json_response['response']['categories'][3]
food_id_dataframe = json_normalize(food_category)
food_id_dataframe = food_id_dataframe[['id', 'name']]

# Filling out the rest of the Food category dataframe with all the children and their respective name and ids
food_categories = food_category['categories']
for i, cat in enumerate(food_categories, 1):
    food_id_dataframe.loc[i] = [cat['id'], cat['name']]

food_id_dataframe.set_index('name', inplace=True)

# Here is the completed food_id_dataframe
food_id_dataframe
# Now to construct the user_feature matrix
user_category_df = food_id_dataframe.T
user_category_df.drop('id', axis=0, inplace=True)
user_category_df
# I need my friends and colleagues to add their user preferences for each restaurant category to the dataframe, so in order to collect their inputs I need an .xlsx version of the user_feature dataframe

writer = pd.ExcelWriter('request_inputs.xlsx', engine='xlsxwriter')
user_category_df.to_excel(excel_writer=writer)

filename = 'request_inputs.xlsx'
with open(filename, 'rb') as z:
        data = io.BytesIO(z.read())
        project.save_data(filename, data, overwrite=True)
# Though the magic of time, my friends and colleagues have input their preferences for each cuisine.
# I gave my friends and colleauges instructions to fill out the user_category matrix with a rank of 1 - 5 (1 being lowest and 5 being highest).
# If my friend didn't have a preference or experience with a particular cuisine, I instructed them the leave the field blank.  The recommender system will take care of that.

# For when you're ready to get rid of that stupid 'Restaurant' category -_-'
# user_category_matrix_column_names = list((food.lower().replace(' ', '_') for food in food_id_dataframe[(food_id_dataframe.index != 'Food') | (food_id_dataframe.index != 'Restaurant')].index))
user_category_matrix_column_names = list((food for food in food_id_dataframe[food_id_dataframe.index != 'Food'].index))
user_category_matrix_column_names.insert(0, 'user_name')
user_category_matrix = pd.DataFrame(columns=user_category_matrix_column_names)

def build_user_category_matrix(user_info_list):
    '''
    Builds the user/category matrix which represents each user's preference for different cuisines
    '''
    user_category_dict = {}
    for user_cat, cat_name in zip(user_info_list, user_category_matrix_column_names):
        user_category_dict[cat_name] = user_cat
    user_category_matrix.loc[len(user_category_matrix)] = user_category_dict

elizabeth = ['Elizabeth', '', 4, 2, 5, 3, '', '', 4, 3, '', 2, 1, 4, 3, 2, 3, 2, 4, 5, '', 2, 2, 3, '', '', 1, 2, 1, 4, '', 3, 2, 3, 1, 1, 1, 2, 1, 4, 2, 1, '', 1, 3, '', 4, '', 4, 1, '', 5, 1, 4, '', 2, 1, 4, 1, 4, 4, 5, '', 1, '', 1, 1, '', '', 1, '', 3, 3, 2, '', '', 1, '', 1, 3, 4, '', 4, 1, '', 1, 1, 1, '', '', 5, 1]
jessica = ['Jessica', '', 3, 2, 4, '', '', 3, 1, 3, 2, 2, 4, 2, 1, 3, 1, 4, 5, 5, '', 5, 5, 2, '', 2, 1, 4, 1, 5, '', '', '', 3, 1, 3, '', 1, 3, 5, 1, 5, '', '', 2, '', 5, 4, 2, 1, '', 5, '', 4, 5, 4, 3, 3, 3, 5, 4, 5, '', 2, 4, 5, 3, '', '', '', '', '', 3, 4, '', '', 3, '', 3, 5, 5, 3, '', 2, '', 3, 5, 1, '', '', 4, 2]
anna = ['Anna', 5, 5, 1, 5, '', '', 4, 1, 1, 4, 3, 5, 5, 3, 4, 1, 5, 5, 5, '', 5, 4, 5, 5, 3, 5, 2, 4, 4, '', 5, 4, 5, 1, 3, 4, 1, 3, 4, 5, 4, 3, 4, 4, 5, 5, 5, '', 1, 3, 5, 3, 3, 5, 4, 5, 5, 1, 5, 5, 5, '', 5, 5, 1, 4, 3, '', 5, '', 4, 4, 1, '', '', 5, 3, 3, 4, 4, 5, '', 4, 4, 5, 3, 1, 5, 3, 3, 3]
candy = ['Candy', '', 1, 5, 5, '', 4, 4, 5, 5, '', 5, 5, 1, 1, 3, 1, 5, 4, '', '', 5, 5, 3, '', 4, 4, 5, 4, 3, '', '', 1, 1, 1, 4, 2, 1, 3, 4, 4, 4, 4, '', 4, 1, 3, 1, 3, 4, 2, 4, '', 5, 4, 1, 1, '', 3, 1, 5, 1, '', '', 1, 5, 5, 1, 1, 1, '', 1, 4, 4, 3, 1, 4, 1, 2, 4, 4, 3, 1, 5, 4, 4, 3, 2, 1, 1, 1, 1]
rosemary = ['Rosemary', 5, 5, 2, 4, '', 3, 1, 4, 3, 2, 4, 1, 5, 2, 1, 3, 4, 2, 3, 3, 4, 3, 5, '', 3, 3, 2, 4, 5, '', 3, 2, 4, 1, 1, 3, 2, 2, 4, 2, 1, 1, 2, 2, 3, 3, 3, 4, 1, 2, 5, 1, 3, 4, 1, 3, 4, 1, 3, 2, 4, 3, 5, 5, 4, 2, 1, '', 3, 4, 3, 3, 3, 4, 2, 5, '', 3, 4, 4, 3, 5, 1, '', 5, 3, 2, 2, 2, 5, 1]
jessie = ['Jessie', 1, 5, 3, 4, 1, 1, 1, 1, 1, 1, 4, 1, 1, 3, 2, 5, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 5, 1, 1, 1, 4, 3, 2, 3, '', '', '', 1, 4, 5, 5, 1, 1, 1, 5, 1, 4, 5, 5, 5, 4, 1, 5, 1, 5, 2, 1, '', '', '', '', '', '', 3, '', 4, '', 1, 1, '', '', '', '', 1, '', 3, '', '', '', '', '', '', '', 5, '']
jonnie = ['Jonnie', '', 4, 5, 4, '', '', 5, 5, 2, '', 3, 5, 1, 3, 5, 2, 2, 4, 1, '', 1, 5, 3, '', 4, 1, 5, 1, 5, '', '', '', 5, 3, 5, 2, 1, 3, 3, '', 4, '', 5, 1, 2, 5, 4, 1, 1, '', 4, 4, 4, 5, 1, 4, 5, 1, 5, 5, 5, '', '', 4, 1, 4, '', '', 4, 5, '', 4, 4, '', '', 5, '', 1, 4, 4, 4, '', 4, '', 2, 2, 1, '', '', 4, 3]
natalie = ['Natalie', 1, 5, 4, 5, 3, 2, 5, 3, 3, 2, 3, 4, 4, 4, 5, 3, 3, 5, 5, 1, 2, 5, 3, 2, 3, 2, 4, 2, 4, 3, 3, 3, 4, 3, 3, 2, 1, 1, 3, 2, 5, 3, 4, 2, 2, 3, 3, 4, 2, 2, 4, 2, 4, 2, 3, 4, 5, 4, 4, 5, 3, 2, 3, 3, 2, 4, 3, 3, 2, 4, 2, 4, 3, 2, 2, 5, 2, 2, 3, 5, 5, 2, 5, 2, 2, 3, 2, 1, 1, 4, 5]
imo = ['Imo', '', 5, 3, 4, 2, '', 3, 4, 5, '', 3, 4, 2, 2, 3, 2, 3, 4, '', '', 3, 4, '', '', '', 4, 4, 3, 3, '', '', 3, 3, 2, 3, '', 3, 3, 2, '', 4, 4, '', 3, 2, '', 3, '', 1, '', 4, '', 3, '', 2, 3, 4, '', 3, 5, 3, 3, '', '', 2, 4, '', '', 4, 3, '', 2, 3, '', '', 1, '', 2, 2, 4, 3, '', 2, '', '', 2, 1, '', '', 3, 3]
eric = ['Eric', '', 3, 4, 4, '', '', 4, 3, 3, 2, 3, 5, 1, 2, 5, 2, 3, 3, 4, '', 2, 4, 2, '', 4, 2, 3, 3, 3, 3, 1, 2, 2, 3, 2, 2, 3, 3, 3, 2, 5, '', '', 2, '', 4, '', 3, 5, '', 5, 2, 5, '', 3, 2, 4, 2, '', 5, 2, '', '', '', '', 5, '', 3, 1, 5, 1, 4, 4, '', 2, 2, '', 3, 4, 5, 3, 2, 5, '', '', 2, 2, '', 2, 2, 5]
shelan = ['Shelan', 2, 5, 5, 5, 3, 3, 4, 2, 5, 3, 4, 5, 5, 3, 4, 2, 4, 3, 4, 3, 3, 3, 4, 2, 2, 4, 4, 4, 3, 3, 3, 4, 4, 3, 4, 3, 2, 3, 4, 4, 4, 3, 3, 3, 2, 4, 4, 4, 3, 2, 2, 4, 5, 2, 3, 4, 4, 3, 4, 5, 4, 4, 3, 3, 2, 4, 3, 3, 2, 4, 2, 3, 3, 2, 2, 1, '', 3, 2, 2, 3, 3, 3, 3, 4, 3, 2, 3, 2, 5, 3]
shari = ['Shari', '', 5, 5, 5, '', '', 5, 3, 3, 5, 4, 5, 5, 3, 4, 1, 3, 5, 3, '', 3, 5, 4, '', 5, 3, 3, 4, 4, '', 4, 1, 1, 2, 2, 3, 2, 2, 5, 3, 5, '', '', '', 1, 4, 3, 3, 3, '', 2, 3, 5, '', 4, 3, 4, 4, 5, 5, '', '', '', '', 3, 3, 3, '', 1, 5, '', 3, 4, '', 1, 4, '', 4, 5, 5, 5, '', 4, '', 3, 1, 1, '', '', 1, 3]
andre = ['Andre', 3, 4, 4, 5, '', 3, 4, 2, 3, 3, 3, 2, 4, 1, 4, 1, 3, 5, 5, 1, 3, 5, 1, '', 3, 1, 2, 1, 5, '', 3, 2, 4, 2, 2, 3, 2, 3, 3, 5, 4, 3, 3, 3, 1, 4, 4, 4, 2, '', 5, 2, 5, '', 2, 4, 5, 1, 5, 5, 4, 3, 3, '', 1, 4, 3, 4, 3, 5, 2, 4, 4, 3, 1, 5, '', 1, 5, 5, 4, '', 3, '', 3, 2, 1, 3, '', 3, 2]
tina_marie = ['Tina-Marie', '', 5, 5, 5, 1, 2, 5, 1, 1, 3, 5, 5, 2, 1, 5, '', 3, 5, 5, '', 4, 4, 4, 2, 5, 5, 5, 2, 3, 3, 2, 1, 5, 3, 4, 5, 2, 3, 3, 2, 3, 2, 3, 1, 5, 5, 4, 2, 1, 1, 4, 2, 4, 3, 5, 5, 5, 4, 5, 4, 4, 2, 2, 3, '', 5, 1, 3, 1, 5, 1, 5, 4, 2, 1, 5, '', 5, 5, 4, 3, '', 4, 2, 4, 2, '', '', '', 4, 4]

# adding the users to the sparse matrix
build_user_category_matrix(elizabeth)
build_user_category_matrix(jessica)
build_user_category_matrix(anna)
build_user_category_matrix(candy)
build_user_category_matrix(rosemary)
build_user_category_matrix(jessie)
build_user_category_matrix(jonnie)
build_user_category_matrix(imo)
build_user_category_matrix(natalie)
build_user_category_matrix(eric)
build_user_category_matrix(shelan)
build_user_category_matrix(shari)
build_user_category_matrix(andre)
build_user_category_matrix(tina_marie)

user_category_matrix.set_index('user_name', inplace=True)
user_category_matrix
# I also have the preferences of the users who submitted profiles

user_pref_column_names = ['user_name', 'loc_name', 'loc_address', 'pref_1', 'pref_2', 'pref_3', 'pref_4', 'pref_5']

user_loc_pref_df = pd.DataFrame(columns=user_pref_column_names)

def add_user_pref(user_list):
    '''
    Adds individual users to the user_loc_pref_df
    '''
    user_dict = {}
    for col, user in zip(user_pref_column_names, user_list):
        user_dict[col] = user
    user_loc_pref_df.loc[len(user_loc_pref_df)] = user_dict

jessica_pref = ['Jessica', 'Amazon Doppler', '2021 7th Ave, Seattle, WA 98121', 'Asian Restaurant', 'Food Truck', 'Greek Restaurant', 'Indian Restaurant', 'Mediterranean Restaurant']
candy_pref = ['Candy', 'The Park Place Building', '1200 6th Ave, Seattle, WA 98101', 'American Restaurant', 'Dumpling Restaurant', 'Italian Restaurant', 'Pizza Place', 'Steakhouse']
jessie_pref = ['Jessie', 'The IBM Building', '1200 5th Ave, Seattle, WA 98101', 'Greek Restaurant', 'Indian Restaurant', 'Mediterranean Restaurant', 'Middle Eastern Restaurant', 'Vegetarian / Vegan Restaurant']
elizabeth_pref = ['Elizabeth', 'Amazon Doppler', '2021 7th Ave, Seattle, WA 98121', 'Asian Restaurant', 'Indian Restaurant', 'Middle Eastern Restaurant', 'Soup Place', 'Vegetarian / Vegan Restaurant']
anna_pref = ['Anna', 'The Walt Disney Company', '925 4th Ave, Seattle, WA 98104', 'Indian Restaurant', 'Mediterranean Restaurant', 'Molecular Gastronomy Restaurant', 'Tea Room', 'Turkish Restaurant']
rosemary_pref = ['Rosemary', 'Westlake Center', '400 Pine St, Seattle, WA 98101', 'African Restaurant', 'Bubble Tea Shop', 'Dumpling Restaurant', 'Seafood Restaurant', 'Vegetarian / Vegan Restaurant']
jonnie_pref = ['Jonnie', 'Impact Hub', '220 2nd Ave S, Seattle, WA 98104', 'Asian Restaurant', 'BBQ Joint', 'Diner', 'Fish & Chips Shop', 'Mexican Restaurant']
natalie_pref = ['Natalie', 'Fred Hutch', '1100 Fairview Ave N, Seattle, WA 98109', 'Wings Joint', 'Mexican Restaurant', 'Gastropub', 'Burger Joint', 'Asian Restaurant']
imo_pref = ['Imo', 'Impact Hub', '220 2nd Ave S, Seattle, WA 98104', 'African Restaurant', 'Asian Restaurant', 'Indian Restaurant', 'Mexican Restaurant', 'Southern / Soul Food Restaurant']
eric_pref = ['Eric', 'King Street Station', '303 S Jackson St, Seattle, WA 98104', 'Fried Chicken Joint', 'Italian Restaurant', 'Mexican Restaurant', 'Southern / Soul Food Restaurant', 'Wings Joint']
shelan_pref = ['Shelan', 'CenturyLink Field', '800 Occidental Ave S, Seattle, WA 98134', 'African Restaurant', 'American Restaurant', 'Caribbean Restaurant', 'Tea Room', 'Vegetarian / Vegan Restaurant']
shari_pref = ['Shari', 'Impact Hub', '220 2nd Ave S, Seattle, WA 98104', 'African Restaurant', 'Asian Restaurant', 'Italian Restaurant', 'Mexican Restaurant', 'Southern / Soul Food Restaurant']
andre_pref = ['Andre', 'Impact Hub', '220 2nd Ave S, Seattle, WA 98104', 'Southern / Soul Food Restaurant', 'Mexican Restaurant', 'Mediterranean Restaurant', 'Indian Restaurant', 'Asian Restaurant']
tina_marie_pref = ['Tina-Marie', 'The Odd Fellows Builiding', '915 E Pine St, Seattle, WA 98122', 'African Restaurant', 'American Restaurant', 'Gluten-free Restaurant', 'Latin American Restaurant', 'Middle Eastern Restaurant']

add_user_pref(jessica_pref)
add_user_pref(candy_pref)
add_user_pref(jessie_pref)
add_user_pref(elizabeth_pref)
add_user_pref(anna_pref)
add_user_pref(rosemary_pref)
add_user_pref(jonnie_pref)
add_user_pref(natalie_pref)
add_user_pref(imo_pref)
add_user_pref(eric_pref)
add_user_pref(shelan_pref)
add_user_pref(shari_pref)
add_user_pref(andre_pref)
add_user_pref(tina_marie_pref)

user_loc_pref_df.set_index('user_name', inplace=True)
user_loc_pref_df
# Now to add the longitude and latitude of each of the users' locations
user_loc_pref_df.index
for user, address in zip(user_loc_pref_df.index, user_loc_pref_df['loc_address']):
    location = geocoder.arcgis(address)
    user_loc_pref_df.at[user, 'loc_lat'] = location.latlng[0]
    user_loc_pref_df.at[user, 'loc_lng'] = location.latlng[1]

user_loc_pref_df
# let's just return a subset of the people we want to be lunch buddies
def select_users(user_list):
    '''
    Returns a subset of the user_loc_pref_df for just the users we want to be lunch buddies
    '''
    return user_loc_pref_df.loc[user_list]

lunch_buddies = ['Elizabeth', 'Jessica', 'Anna', 'Candy', 'Rosemary', 'Jessie', 'Jonnie', 'Natalie', 'Imo', 'Eric', 'Shelan', 'Shari', 'Andre', 'Tina-Marie']
buddies_df = select_users(lunch_buddies)
buddies_df
# Let's find the central location of the lunch buddies.  This will be done by getting the mean latitude and longitude of all the users.
def get_central_loc(user_list):
    '''
    Returns the central location of the lunch buddies
    '''
    central_lat = user_loc_pref_df['loc_lat'][user_list].mean()
    central_lng = user_loc_pref_df['loc_lng'][user_list].mean()
    return(central_lat, central_lng)

central_location = get_central_loc(lunch_buddies)
central_location
def get_lunch_buddy_sparse_matrix(user_list):
    '''
    Returns the sparse matrix of the selected lunch buddies
    '''
    return user_category_matrix.loc[user_list]

buddy_sparse_matrix = get_lunch_buddy_sparse_matrix(lunch_buddies)
buddy_sparse_matrix
# What to do about people at the same location?  Let's group them for mapping purposes
def check_for_dupe_locs():
    '''
    Checks for users in the user_loc_pref_df who are located at the same location
    '''
    users_at_same_loc = buddies_df.loc[buddies_df.duplicated(subset=['loc_lat', 'loc_lng'], keep=False), :]
    if len(users_at_same_loc) > 0:
        users_at_same_loc = users_at_same_loc[['loc_lat', 'loc_lng', 'loc_name', 'loc_address']]
        users_at_same_loc = users_at_same_loc.groupby(['loc_lat', 'loc_lng', 'loc_name', 'loc_address'], as_index=False).apply(lambda x: ' and '.join(x.index))
        if isinstance(users_at_same_loc.unstack().loc[users_at_same_loc.unstack().index[0], users_at_same_loc.unstack().columns[0]], float):
            return users_at_same_loc.unstack().loc[::,::-1]
        else:
            return users_at_same_loc.unstack()
    else:
        return None

users_at_same_loc = check_for_dupe_locs()

for i, address in enumerate(users_at_same_loc):
    buddies = users_at_same_loc.loc[:, address][i] # access the lunch buddies
    lat = users_at_same_loc.index[i][0] # access the lat of the index
    lng = users_at_same_loc.index[i][1] # access the lng of the index
    loc = users_at_same_loc.index[i][2] # access the loc of the index
    print(f'{buddies} are at {loc}, which is located at ({lat}, {lng}).')
# Let's map out Seattle, WA USA
seattle_wa = geocoder.arcgis('Seattle, WA')

seattle_lat, seattle_lng = seattle_wa.latlng[0], seattle_wa.latlng[1]
seattle_map = folium.Map(location=(seattle_lat, seattle_lng), zoom_start=14)

def create_initial_map(user_dataframe):
    '''
    Creates an initial map of where the lunch buddies are located
    '''
    for lat, lng, uname, locname in zip(user_dataframe['loc_lat'], user_dataframe['loc_lng'], user_dataframe.index, user_dataframe['loc_name']):
            folium.CircleMarker(location=(lat, lng), popup= f'{uname} is located at {locname}').add_to(seattle_map)
    if len(users_at_same_loc) > 0:
        for i, address in enumerate(users_at_same_loc):
            buddies = users_at_same_loc.loc[:, address][i] # access the lunch buddies
            lat = users_at_same_loc.index[i][0] # access the lat of the index
            lng = users_at_same_loc.index[i][1] # access the lng of the index
            loc = users_at_same_loc.index[i][2] # access the loc of the index
            folium.CircleMarker(location=(lat, lng), popup= f'{buddies} are located at {loc}').add_to(seattle_map)

    folium.CircleMarker(location=(central_location[0], central_location[1]), popup=f'This is the central location', color='green').add_to(seattle_map)

create_initial_map(buddies_df)
seattle_map
def create_target_users_score_matrix(user_dataframe):
    '''
    Creates a matrix of each lunch buddy's cuisine preference score
    '''
    pref_list = []
    prefs_df = user_dataframe[['pref_1', 'pref_2', 'pref_3', 'pref_4', 'pref_5']]
    for user in user_dataframe.index:
        for pref in prefs_df.columns:
            pref_list.append(prefs_df.loc[user, pref])

    pref_set = set(pref_list)
    pref_list = list(pref_set)
    user_list = user_dataframe.index.to_list()
    
    user_pref_score_matrix = pd.DataFrame(columns=pref_list, index=user_list)
    
    for user in user_list:
        for pref in pref_list:
            users_preference = user_category_matrix.loc[user, pref]
            user_pref_score_matrix.loc[user, pref] = users_preference
    
    return user_pref_score_matrix

user_score_matrix = create_target_users_score_matrix(buddies_df)
user_score_matrix
def create_composite_user(score_matrix):
    '''
    Returns a series of the composite ranking of each cusine's ranking by all of the lunch buddies.
    '''
    score_matrix = score_matrix.replace('', 0)
    return score_matrix.sum(axis=0).sort_values(ascending=False)

composite_user = create_composite_user(user_score_matrix)
composite_user
# Let's get a nice silhouette image
# IMPORTANT!  THE MASK HAS TO BE A BLACK AND WHITE IMAGE!  NO COLORS!  NO TRANSPARENT BACKGROUNDS!
# THE WORD CLOUD WILL BE RENDERED ONTO THE BLACK PARTS OF THE IMAGE!
! wget --quiet https://image.freepik.com/free-icon/oriental-bowl-chopsticks_318-9695.jpg -O bowl_silhouette.jpg
! wget --quiet https://png2.cleanpng.com/sh/ad2e1971b50358d76e8bc84b657a7922/L0KzQYm3VsEzN5lwipH0aYP2gLBuTfZwepwyg9DyZnWwc8b7jPVzgV51hNN9ZT31f8rojL1kaaNuetRuYX6wc8P8igNme146eqNvNEi6c7SCUsg3PF83UKcBOUW7RYK8Usg4P2c9SqoDMkSxgLBu/kisspng-fork-knife-cutlery-plate-royal-caribbean-cruises-5b1f487cc92864.285695851528776828824.png -O fork_and_knife.png
! wget --quiet https://static.vecteezy.com/system/resources/previews/000/532/635/large_2x/vector-waiter-waitress-serve-icon-dish-up.jpg -O waiter.jpg
! wget --quiet https://res.cloudinary.com/teepublic/image/private/s--Am50TV5---/c_crop,x_10,y_10/c_fit,w_1109/c_crop,g_north_west,h_1260,w_1260,x_-76,y_-94/co_rgb:ffffff,e_colorize,u_Misc:One Pixel Gray/c_scale,g_north_west,h_1260,w_1260/fl_layer_apply,g_north_west,x_-76,y_-94/bo_157px_solid_white/e_overlay,fl_layer_apply,h_1260,l_Misc:Art Print Bumpmap,w_1260/e_shadow,x_6,y_6/c_limit,h_1134,w_1134/c_lpad,g_center,h_1260,w_1260/b_rgb:eeeeee/c_limit,f_jpg,h_630,q_90,w_630/v1547259589/production/designs/3947399_0.jpg -O steaming_bowl.jpg

! ls -a
def cuisine_word_cloud(composite):
    food_word_cloud_mask = np.array(Image.open('bowl_silhouette.jpg'))
    stop_words = (['Restaurant', 'Joint', 'Shop', 'Spot', 'Food', 'Place'])
    composite_word = ''
    
    word_cloud_array = []
    composite_dict = composite.to_dict()
    for key, value in composite_dict.items():
        for i in range(value):
            word_cloud_array.append(key)
    composite_word = ' '.join(word_cloud_array)
    
    food_word_cloud = WordCloud(background_color='white',
                                stopwords=stop_words,
                                colormap=plt.cm.prism)
#                                 mask=food_word_cloud_mask, )

    food_word_cloud.generate(composite_word)

    fig = plt.figure(figsize=(14,18))

    plt.imshow(food_word_cloud, interpolation='bilinear')
    plt.axis('off')
    plt.show()
    
cuisine_word_cloud(composite_user)
# Who would make the best lunch buddies?  Let's explore that question by visualizing who has the most in common with each others' tastes
def create_user_score_matrix(score_matrix):
    '''
    Creates the full cosine similarity matrix for all the lunch buddies.
    '''
    full_score_matrix = score_matrix.replace('', 0)
    buddy_csr = csr_matrix(full_score_matrix.values)
    similarity_matrix = cosine_similarity(buddy_csr)
    return similarity_matrix

def render_cosine_similarity_matrix(similarity_matrix):
    '''
    Renders a heatmap of the similarity score matrix.
    '''
    fig = plt.figure(figsize=(15,15))
    sns.heatmap(similarity_matrix, 
                xticklabels=user_score_matrix.index,
                yticklabels=user_score_matrix.index,
                cmap='RdBu_r',
                annot=True,
                linewidth=0.5)
    plt.show()
    
user_similarity_matrix = create_user_score_matrix(user_score_matrix)
render_cosine_similarity_matrix(user_similarity_matrix)
# Get the composite user's top 5 picks
test = composite_user.groupby(composite_user.values, sort=True).apply(lambda x: ', '.join(x.index))
test_joined = pd.Series(test.index, test.values)
top_5 = test_joined.nlargest(n=5)

all_cuisines = []
for list_item in top_5.index.tolist():
    if ', ' in list_item:
        each_item = list_item.split(', ')
        all_cuisines.extend(each_item)
    else:
        all_cuisines.append(list_item)

all_cuisines
# Use the FourSquare API to return the composite user's top 5 picks
category_ids = []
for cuisine in all_cuisines:
    cuisine_id = food_id_dataframe[food_id_dataframe.index == cuisine]['id'][0]
    category_ids.append(cuisine_id)
category_ids
# Let's see them all together!
for cuisine, category in zip(all_cuisines, category_ids):
    print(f'{cuisine} has a category id of {category}')
# Now that we have the central location, we can make calls to the FourSquare API to explore which restaurants are in the area around the central location
radius = 100 # in meters
limit = 10

cat_id = food_id_dataframe[food_id_dataframe.index == 'Food']['id'].values[0]

# Creating the url for the categories API call
food_venues_url = f'https://api.foursquare.com/v2/venues/search?client_id={CLIENT_ID}&client_secret={CLIENT_SECRET}&v={VERSION}&ll={central_location[0]},{central_location[1]}&radius={radius}&categoryId={cat_id}&limit={limit}'
food_venues_url

# Creating the nub of the Food category dataframe with the parent cat name and id
food_response = requests.get(food_venues_url).json()
sub_cats = food_response['response']['venues']
for cat in sub_cats:
    print(cat['id'], cat['name'], cat['location']['lat'], cat['location']['lng'], cat['location']['address'], \
          [c['id'] for c in cat['categories']], \
          [c['name'] for c in cat['categories']], \
          [c['icon']['prefix']+'bg_32'+c['icon']['suffix'] for c in cat['categories']][0])
# venues_dataframe = json_normalize(cat['id'], cat['name'], cat['location']['lat'], cat['location']['lng'], cat['location']['address'], [c['id'] for c in cat['categories']], [c['name'] for c in cat['categories']], [c['icon']['prefix']+'bg_44'+c['icon']['suffix'] for c in cat['categories']][0])
# venues_dataframe = food_id_dataframe[['id', 'name']]


# Explore the VENUEID endpoint to see if it's any different than the CATEGORY endpoint.  If they're the same, just use the information in CATEGORY.
# Add the results to the exisiting map

# use when mapping out the venues    
# icon = folium.features.CustomIcon('https://ss3.4sqi.net/img/categories_v2/food/cafe_bg_32.png', icon_size=(32, 32))
# folium.Marker(location=(central_location[0], central_location[1]), popup=f'This is the central location', icon=icon, color='green').add_to(seattle_map)
# Turn the sparse matrix into a fully populated matrix with lightfm
user_preference_matrix = user_score_matrix.replace('', 0)
user_preference_csr = csr_matrix(user_preference_matrix)
user_preference_csr

# model = LightFM(loss='warp')
# %time model.fit(user_preference_csr, epochs=30, num_threads=2)

# n_users, n_items = user_preference_csr.shape

# user_ids = user_preference_csr.data
# user_ids

# for user_id in ['Jessica', 'Jessie']:
#     known_positives = user_preference_csr[user_preference_csr.columns.tocsr()[user_id].indices]
#     print(known_positives)

# model.predict(['Jessica'])

# from lightfm.datasets import fetch_movielens

# data = fetch_movielens(min_rating=5.0)

# print(repr(data['train']))
# print(repr(data['test']))

# model.fit(data['train'], epochs=30, num_threads=2)

# data.indices

# print("Train precision: %.2f" % precision_at_k(model, data['train'], k=5).mean())
# print("Test precision: %.2f" % precision_at_k(model, data['test'], k=5).mean())

# def sample_recommendation(model, data, user_ids):
    

#     n_users, n_items = data['train'].shape

#     for user_id in user_ids:
#         known_positives = data['item_labels'][data['train'].tocsr()[user_id].indices]
        
#         scores = model.predict(user_id, np.arange(n_items))
#         top_items = data['item_labels'][np.argsort(-scores)]
        
#         print("User %s" % user_id)
#         print("     Known positives:")
        
#         for x in known_positives[:3]:
#             print("        %s" % x)

#         print("     Recommended:")
        
#         for x in top_items[:3]:
#             print("        %s" % x)
        
# sample_recommendation(model, data, [3, 25, 450])
# Group the lunch buddies into k-neighbors groups (check for best k)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.15237728180512244, 'fraction_numerical': 0.05157492729757191, 'mean_word_length': 4.348950524737631, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 7, 'https://': 7, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 7, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '376347', 'license': 'MIT', 'n_tokens_mistral': 11886, 'n_tokens_neox': 10272, 'n_words': 3283, 'path': 'I_Dont_Know_Where_Do_You_Want_to_Eat.ipynb', 'repo_name': 'Astayonix/BattleOfTheNeighborhoods', 'size': '28541'}