<jupyter_start><jupyter_text>General Notes  This analysis was completed over several days.  To minimize the re-running all the previous steps, incremental versions of the data was saved.  Throughout the notebook you'll see places to load the "Clean & Enriched" data frame from storage.  Similarly, you'll see many statements commented out as not all cleansing steps were required on a given run of analysis. If something doesn't quite make sense, it is most likely the result of loading the DataFrame from storage and skipping unecessary steps.  Data Collection Local High School On-Line Schedule**Notes**    - The 2019-2020 schedule is not online yet, so using last years schedule to test & build the  model.  - The school selected is an Independent, College Prepratory school, near to where I live. Prep the Sporting Event Location Data1.  Web-Scrape the Schedule for Location Names  1.  Enrich Location Names with Geocode Data  1.  Generate high-level travel route data based on location  1.  Store Intermediary DataFrame so don't need to continue to web-scrape and call API's    **NOTES**  - In Future runs, jump down to where DataFrame is loaded from CSV.<jupyter_code>schedule_url = 'https://www.cushing.org/team-detail?fromId=204845&Team=87969&SeasonLabel=2018+-+2019'
_headers = {
    'User-Agent': 'My User Agent 1.0'
}
r = requests.get(schedule_url, headers=_headers )
# print (r.text)
html_soup = BeautifulSoup(r.text, 'html.parser')
det_containers = html_soup.find_all('div', class_='game')
web_event_data = [[s for s in containers.stripped_strings] for containers in det_containers]
# Loop through web-data, make a dictionary for each game listed on the website
sporting_event_list = []
for game in web_event_data:
    idx = 0
    sporting_event = {}
    sporting_event['date'] = game[idx]
#    date_list.append(sporting_event['date'])  # create separate list of values to c
    idx += 1
    sporting_event['time'] = game[idx]
    idx += 1
    sporting_event['home_away'] = game[idx]
    idx += 1
    if game[idx] == 'Scrimmage':
        idx += 1
    if game[idx] == 'vs.':
        idx += 1 # skip 'vs.'
    sporting_event['opponent_school'] = game[idx]
    idx += 1
    sporting_event_list.append(sporting_event)<jupyter_output><empty_output><jupyter_text>From that list of games, make a pandas DataFram<jupyter_code>games_df = pd.DataFrame(sporting_event_list)
games_df.head()<jupyter_output><empty_output><jupyter_text>Starting Geocoding based on school name   **NOTE:** - It is common to play the same opponent year after year.- As a result, the analysis will be completed for all opponents, but final list of analysis, will only include 'travel sites' (i.e. "Away" games)First step is to set the **HOME** Location, so there is a starting point when calculating distances.<jupyter_code>home_location_name = 'Cushing Academy'
g = geocoder.osm(home_location_name)
if not g.ok:
    print ('GeoCode Failed')
    
home_location = {'quality': g.json['quality'], 'address': g.json['address'] , 'state': g.json['state'], 'lat': g.json['lat'], 'lng': g.json['lng']}
print(home_location)<jupyter_output>{'quality': 'school', 'address': 'Cushing Academy, 39, School Street, Ashburnham, Worcester County, Massachusetts, 01430, United States of America', 'state': 'Massachusetts', 'lat': 42.63286695, 'lng': -71.9135057096491}<jupyter_text>---   Geocoding helper functions  Define functions to make data processing easier- Function `get_geo_info`    Looks up the latitude and longitude for a school based on school name and returns a Pandas Series, so it's easier to build build a DataFrame while looping through list of school names.   This function also tries multiple geocoding services to find latitude and longitude.  It first starts with Open Street Maps Nominatim, if no results are found, it tries the free version of the ArcGIS service.    - Function `clean_row_geo_info`    A helper function to correct coordinates in the event location list.  Pass in the index of the row to correct in teh DataFrame, along with the preferred search terms, and it will correct the DataFrame.    - Function `get_route_data`    A helper function to get the route information between two sets of coordinates.  The service is basically a "Demo" service, so fold int logic to handle multiple retries, and smart backoff logic to increase liklihood the server will be able able to process the routing requests.       Using Project OSRM Routing Service  `http://router.project-osrm.org/route/v1/driving/-71.91356999999994,42.63187000000005;-71.6747789477679,43.76177855`     - Function `convert_meters_to_miles`    The routing service returns driving distance between two coordinate points in meters.  This converts meters to miles.      Meters to Miles conversion = meters * 0.00062137    - Function `convert_seconds_to_duration_str`    The routing service returns driving duration between two coordinate points in seconds.  This converts seconds into an `hh:mm` string to represent the duration.<jupyter_code>## Define Function to look-up Geocode by school name
def get_geo_info(_name):
    g = geocoder.osm(_name)
    if not g.ok:
        # Try another Geocoder
        print ('trying ArcGIS for {}'.format(_name))
        g = geocoder.arcgis(_name)
        if not g.ok:
            print ('Had a problem with {}'.format(_name))
            return pd.Series({'quality': 'error', 'address': 'error', 'state': 'error', 'lat': 0, 'lng': 0})
        # arcgis does not have as many fields     
        return pd.Series({'quality': g.json['quality'], 'address': g.json['address'] , 'state': 'n/a', 'lat': g.json['lat'], 'lng': g.json['lng']})

    
    return pd.Series({'quality': g.json['quality'], 'address': g.json['address'] , 'state': g.json['state'], 'lat': g.json['lat'], 'lng': g.json['lng']})
## Clean-Up Function ... some Geocodes matched the wrong school
def clean_row_geo_info(_dataframe, _df_index, _better_search_name):
    g = geocoder.osm(_better_search_name)
    #print ('old value', _dataframe.iloc[_df_index])
    geo_data = None
    if not g.ok:
        # Try another Geocoder
        print ('trying ArcGIS for {}'.format(_better_search_name))
        g = geocoder.arcgis(_better_search_name)
        if not g.ok:
            print ('Had a problem with {}'.format(_better_search_name))
            return 
        # arcgis does not have as many fields     
        geo_data = {'quality': g.json['quality'], 'address': g.json['address'] , 'state': 'n/a', 'lat': g.json['lat'], 'lng': g.json['lng']}
    else:
        geo_data = {'quality': g.json['quality'], 'address': g.json['address'] , 'state': g.json['state'], 'lat': g.json['lat'], 'lng': g.json['lng']}
    
    _dataframe.at[_df_index, 'quality'] = geo_data['quality']
    _dataframe.at[_df_index, 'address'] = geo_data['address']
    _dataframe.at[_df_index, 'state'] = geo_data['state']
    _dataframe.at[_df_index, 'lat'] = geo_data['lat']
    _dataframe.at[_df_index, 'lng'] = geo_data['lng']   
    #print ('new value', _dataframe.iloc[_df_index])

    
    return
    
@backoff.on_exception(backoff.expo,
                      requests.exceptions.RequestException,
                      max_tries=8,
                     jitter=backoff.full_jitter)    
def get_route_data(_row):
    url = 'http://router.project-osrm.org/route/v1/driving/{},{};{},{}'.format(home_location['lng'], home_location['lat'],
                                                                              _row['lng'], _row['lat'])
    
    r = requests.get(url)
    r.raise_for_status()  # Ugly, but blowup if web-request doesn't work
    r_data = r.json()
#    print (r.text)
    print ( _row['opponent_school'], r_data['code'])  # make sure this is 'Ok'
    print (datetime.utcnow())  # Just checking backoff :) 
    if len(r_data['routes']) > 0:
        r_data = r_data['routes'][0] # Just grab first available route ...
    else:
        print (r_data['code'], _row['opponent_school'])
        return pd.Series({'distance_meters': 0, 'duration_seconds': 0})
    
    return pd.Series({'distance_meters': r_data['distance'], 'duration_seconds': r_data['duration']})
    
def convert_meters_to_miles(_meters):
    return _meters * 0.00062137

def convert_seconds_to_duration_str(_seconds):
    return str(timedelta(seconds=_seconds))[0:4]  # only show HH:MM  in e<jupyter_output><empty_output><jupyter_text>Actually do the geocoding work Using the helper functions above, add data from geocoding services to the list of sporting events locations (i.e. the games)<jupyter_code># Get Lat & Long for school names
games_df[["quality", "address", "state", "lat", "lng"]] = games_df['opponent_school'].apply(get_geo_info)<jupyter_output>trying ArcGIS for Dexter Southfield School<jupyter_text>Data Cleansing Iterative run & re-run the following few cells, to clean-up the data that needs it, due to incorrect GEOCode Searches.  The first cell is to verify the results, the cell after is actually the clean-up functions.Commenting out the statements after they've cleansed the data, but keeping them in the cell for future reference.  (But after youyou fixed the data, don't need to keep fixing it)<jupyter_code>#  Check the States, make sure all the Geocode results look OK
games_df['state'].value_counts()

# Interrogate unexpected  records
#games_df.loc[games_df['state'] == 'Pennsylvania']
#games_df.loc[games_df['state'] == 'n/a']
#games_df.loc[games_df['state'] == 'North Carolina']


# Clean-Up Records
#clean_row_geo_info(games_df, 12, "Brooks School, North Andover, MA")
#clean_row_geo_info(games_df, 14, "St. George's School, RI, US")
#clean_row_geo_info(games_df, 18, "Lawrence Academy, MA, US")
#clean_row_geo_info(games_df, 21, "Dexter Southfield, Brookline, MA, 02445, US")
#clean_row_geo_info(games_df, 22, "St. Paul's School, NH, US")
# games_df.at[21, 'state'] = 'Massachusetts'   # ArcGIS doesn't provide state, but I know it<jupyter_output><empty_output><jupyter_text>Just as a nice to have, add route dataFetch driving routes, based on coordinates, to get distance & durationAnd then transform from meters & seconds to miles & string of the form "hh:mm"<jupyter_code># get Driving Distance & Duration from "home school" (Cushing Academy) to each school
games_df[['distance_meters', 'duration_seconds']] = games_df.apply(get_route_data, axis=1)
print ('route data loaded')
games_df.head()
# Convert the Routing information into differt units
#games_df['distance_miles'] = games_df['distance_meters'].apply(convert_meters_to_miles)
#games_df['duration_str'] = games_df['duration_seconds'].apply(convert_seconds_to_duration_str)

games_df.describe()
max_distance = games_df['duration_seconds'].idxmax()
print ('The Median Travel Distance is: %.1f miles' % games_df['distance_miles'].median())
print ('The Median Travel Duration is:', convert_seconds_to_duration_str(5013.800000))
print ('The game at {} is the farthest drive, at %.1f miles'.format(games_df.iloc[max_distance]['opponent_school']) % games_df.iloc[max_distance]['distance_miles'])<jupyter_output><empty_output><jupyter_text>Save the Games DataFrameNow that the Games (sporting event location) DataFrame is cleansed & enriched, save it to a CSV, so you can jump right here and just reload it.(before getting started on the Foursquare work<jupyter_code># Save games_df form project files

# There are some additional enrichments below, made to the games_loc_df, and I want to persist the refined enrichmetns between runs.
# --> So choose one of the DataFrames to store... 

#project.save_data("game_location_data.csv", games_df.to_csv(index=False))

project.save_data("game_location_data.csv", games_loc_df.to_csv(index=False), overwrite=True)<jupyter_output><empty_output><jupyter_text>--- Load the Location DataFrame from CSV to get started with Foursquare API analysisPicking up with analysis, and starting with clean event location data.The next few cells loads the Games Locations DataFrame and explores and summarizes the data<jupyter_code>games_loc_df = pd.read_csv(project.get_file('game_location_data.csv'))
games_loc_df.head()
games_loc_df.shape
games_loc_df.groupby(['home_away']).sum()

games_loc_df.groupby(['home_away']).agg({'distance_miles': ['sum', 'mean', 'median'],
                                        'duration_minutes': ['sum', 'mean', 'median']})
print ('Total roundtrip miles traveled for 13 away games was {:,.1f} miles with an estimated duration of {}'.format(872.660541 * 2., str(timedelta(seconds=68683.9 * 2.))[:-10]))
print ('The average one-way trip distance was {:,.1f} miles for the {} Away games'.format(games_loc_df.loc[games_loc_df['home_away'] == 'Away']['distance_miles'].mean(),
                                                                                  games_loc_df.loc[games_loc_df['home_away'] == 'Away']['distance_miles'].count()) )<jupyter_output>Total roundtrip miles traveled for 13 away games was 1,745.3 miles with an estimated duration of 1 day, 14:09
The average one-way trip distance was 67.1 miles for the 13 Away games<jupyter_text>---  ---   Venue Related Data Collection Now on to Foursquare / Venue Data Collection-  Limit Foursquare search to 100 venues per sporting event location  -  The Search Radius is 2.5 miles around any event location Some Foursquare Helper Functions  - Function `getNearbyVenues`    Use the Foursquare `Explore` API endpoint to fetch the top 100 most recommended venue with a radius of a given location.    - Function `get_venue_rating`    Use the Foursquare API to get additional information such as `rating`.  **Note** this is a _premium_ call.  As such Foursquare limits the amount of venue information you can access free-of-charge.  As a result, rating data is only a _nice to have_, and not every venue in the analysis has rating data.        This function tries to be economical with API access, so if it detects an existing rating, it will skip that record and move on to the next, in an attempt to get maximum number of venues with venue rating information.      - Function `get_parent_category`    For a given category return the top level parent for that category.  Example `Multiplex` is a child of `Movie Theater` which is a child of `Arts & Entertainment`  In this case, you pass in the category id for `Multiplex` and you are returned `Arts & Entertainment`      - Function `traverse_children`    Categories in Foursquare are stored in a tree structure.  This function traverses the tree and builds a 1:1 mapping of all categories to their top-level parent.  The results of this traversal are used by `get_parent_category` function.<jupyter_code># Helper Function to Process list of locations

def getNearbyVenues(names, latitudes, longitudes, radius=500):
    
    venues_list = []
    meta_result_list = []
    for name, lat, lng in zip(names, latitudes, longitudes):       
            
        # create the API request URL
        url = 'https://api.foursquare.com/v2/venues/explore?&client_id={}&client_secret={}&v={}&ll={},{}&radius={}&limit={}'.format(
            CLIENT_ID, 
            CLIENT_SECRET, 
            VERSION, 
            lat, 
            lng, 
            radius, 
            LIMIT)
            
        # make the GET request
        r = requests.get(url).json()["response"]
        results = r['groups'][0]['items']
        print(name, r['totalResults'])
        meta_result_list.append((name, r['totalResults']))
            
        # return only relevant information for each nearby venue
        venues_list.append([(
            name, 
            lat, 
            lng, 
            v['venue']['name'], 
            v['venue']['location']['lat'], 
            v['venue']['location']['lng'],  
            v['venue']['categories'][0]['name'],
            get_parent_category(v['venue']['categories'][0]['id']),
            
            v['venue']['id']) for v in results])

    nearby_venues = pd.DataFrame([item for venue_list in venues_list for item in venue_list])
    nearby_venues.columns = ['Event Location', 
                  'Event Location Latitude', 
                  'Event Location Longitude', 
                  'Venue', 
                  'Venue Latitude', 
                  'Venue Longitude', 
                  'Venue Category',
                  'Venue Root Category',                             
                  'Venue Id']
    
    return(nearby_venues, meta_result_list)


def get_venue_rating(_venue_info):
    """ Foursquare has daily limit of 500 premium calls ... pass in existing rating and skip when it has data, to incrementally build dataset"""
    _venue_id = _venue_info['Venue Id']
    _existing_rating = _venue_info['rating']
    if _existing_rating > 0:
        return _existing_rating
    
    _params = {
        'client_id': CLIENT_ID,
        'client_secret': CLIENT_SECRET,
        'v': VERSION,
        'limit': 1
    }
    ven_dets_url = 'https://api.foursquare.com/v2/venues/%s' % _venue_id
    r = requests.get(ven_dets_url, params=_params)
    if r.status_code != 200:
        print ('Issue with ', _venue_id)
        print (r)
        return 0
    venue_data = r.json()['response']['venue']
    
    if 'rating' in venue_data:
        return venue_data['rating']
    else:
        return 0

def get_parent_category(_category_id):
    """ mapping of all Foursquare Categories to their Parent category """
    if _category_id in parent_category_map:
        return parent_category_map[_category_id]['my_top_parent'][0]
    else:
        'Unkown'
    

def traverse_children(_parent_category_map, _eldest_parent, _cat_list):
    """ Recursively traverse the Foursquare list of categories mapping to top parent"""    

    item_count = len(_cat_list)

    if item_count == 0:
        return
    
    else:        
        c = _cat_list[0]  
        _parent_category_map[c['id']] = {'my_name': c['name'], 'my_top_parent': _eldest_parent}
        traverse_children(_parent_category_map, _eldest_parent, _cat_list[1:])

        if len(c['categories']) > 0:
            traverse_children(_parent_category_map, _eldest_parent, c['categories'])<jupyter_output><empty_output><jupyter_text>**Build Foursquare category Map**Map each category (id) to the top-level category, to enable doing analysis of "Food" instead of "Wings Joint"<jupyter_code>_params = {
    'client_id': CLIENT_ID,
    'client_secret': CLIENT_SECRET,
    'v': VERSION,
    'limit': 1
}
fsapi_url = 'https://api.foursquare.com/v2/venues/categories'
r = requests.get(fsapi_url, params=_params)
r.raise_for_status()
cat_data = r.json()['response']
parent_list = cat_data['categories']

parent_category_map = {}

print (len(parent_list), 'Top Level Categories\n', [r['name'] for r in parent_list])
for root_node in parent_list:
    elder_dets = (root_node['name'], root_node['id'])
    traverse_children(parent_category_map, elder_dets, root_node['categories'])
    
# Quick Test, 'Pizza Place'
print (parent_category_map['4bf58dd8d48988d1ca941735'])
print ('Number of categories in map', len(parent_category_map.keys()))<jupyter_output>10 Top Level Categories
 ['Arts & Entertainment', 'College & University', 'Event', 'Food', 'Nightlife Spot', 'Outdoors & Recreation', 'Professional & Other Places', 'Residence', 'Shop & Service', 'Travel & Transport']
{'my_name': '<NAME>', 'my_top_parent': ('Food', '4d4b7105d754a06374d81259')}
Number of categories in map 927<jupyter_text>Get Venue Data from Foursquare  **Reminder**  The core venue analysis is limited to 100 venues per location.  However, the Foursquare API returns the total number of venues in the result set.  The total number of venues is interesting metadata related to each location.  So store the total number of venues per location and use it to enrich the game location DataFrame<jupyter_code>search_radius = 2.5 * 1609.344  # 2.5 miles x conversion = 4,023.36
venues_df, high_level_venue_counts = getNearbyVenues(games_loc_df['opponent_school'], games_loc_df['lat'], games_loc_df['lng'], search_radius)
venues_df.shape
print(high_level_venue_counts)   #  Grab total number of venues b/c result set was limited to top 100 venues at each location<jupyter_output>[('Middlesex School', 16), ('Dana Hall School', 87), ('Buckingham Browne & Nichols', 239), ('Holderness School', 41), ('Vermont Academy', 3), ("Miss Porter's school", 62), ('Phillips Exeter Academy', 72), ('Williston Northampton School', 60), ('Winchendon School', 20), ('Pomfret School', 12), ("St. Mark's School", 17), ("Governor's Academy", 12), ('Brooks School', 47), ('Wilbraham & Monson Academy', 22), ("St. George's School", 191), ('Brewster Academy', 31), ('Kimball Union Academy', 8), ('Northfield Mt. Hermon School', 10), ('Lawrence Academy', 26), ('Groton School', 41), ("Miss Hall's School", 95), ('Dexter Southfield School', 232), ("St. Paul's School", 55)]<jupyter_text>**Enrich the Games Location DataFrame**  Enrich the Event Location dataset with total number of venues near each location.<jupyter_code># Use Text Output to create hard-coded list, to avoid hitting API ...
venue_count_list = [('Middlesex School', 16), ('Dana Hall School', 87), ('Buckingham Browne & Nichols', 239), ('Holderness School', 41), ('Vermont Academy', 3), ("Miss Porter's school", 62), ('Phillips Exeter Academy', 72), ('Williston Northampton School', 60), ('Winchendon School', 20), ('Pomfret School', 12), ("St. Mark's School", 17), ("Governor's Academy", 12), ('Brooks School', 47), ('Wilbraham & Monson Academy', 22), ("St. George's School", 191), ('Brewster Academy', 31), ('Kimball Union Academy', 8), ('Northfield Mt. Hermon School', 10), ('Lawrence Academy', 26), ('Groton School', 41), ("Miss Hall's School", 95), ('Dexter Southfield School', 232), ("St. Paul's School", 55)]

games_loc_df['Nearby Venue Count'] = [x[1] for x in venue_count_list]
games_loc_df['duration_minutes'] = games_loc_df['duration_seconds'].apply(lambda x: x / 60)

games_loc_df.head()

# --> Jump back a few cells and Save the games_loc_df DataFrame again, with the summary venue counts added<jupyter_output><empty_output><jupyter_text>Additional Enrichement of Venue Data**How close are the Venues to the Sporting Event Location ?**  Calculate [Geodesic distance](https://en.wikipedia.org/wiki/Geodesics_on_an_ellipsoid) from each venue to the Sporting Event Location ... i.e. as the 'crow flies'<jupyter_code>venues_df['Geodesic Distance From Event'] = venues_df.apply(lambda x:   geodesic((x['Event Location Latitude'], x['Event Location Longitude']), 
                                                                                 (x['Venue Latitude'], x['Venue Longitude'])).miles, axis=1)


print(venues_df.shape)
venues_df.head()<jupyter_output>(1037, 10)<jupyter_text>What is the quality of these venues ?Fetch ratings for each venueNeed to do this in multiple passes, so store ratings_df DataFrame and merge in<jupyter_code># Load previous ratings dataset

ratings_df = pd.read_csv(project.get_file("venue_ratings.csv"))
print (ratings_df.shape)
ratings_df.head()

ratings_df['rating'] = ratings_df[['Venue Id', 'rating']].apply(get_venue_rating, axis=1)
ratings_df.loc[ratings_df['Venue Id'] == '4abd58d7f964a520da8920e3']
# Save a copy of the venue ratings data
project.save_data("venue_ratings.csv", ratings_df.drop_duplicates(subset=['Venue Id']).to_csv(index=False), overwrite=True)
print ('file saved at', datetime.utcnow())
# How Many Ratings are in the dataset  
ratings_df['rating'].hist()<jupyter_output><empty_output><jupyter_text>Save the Venue data .. since there API Limits, and analysis might span several days, save current state so can pick up where left off<jupyter_code>project.save_data("venue_data.csv", venues_df.to_csv(index=False), overwrite=True)
print (datetime.utcnow()) 
# Load Venues DataFrame from Project Files
venues_df = pd.read_csv(project.get_file("venue_data.csv"))
print (venues_df.shape)
venues_df.head()


# Spot check a venue
venues_df.loc[venues_df['Venue Id'] == '4abd58d7f964a520da8920e3']<jupyter_output><empty_output><jupyter_text>Merge Ratings with Venue Datacreate `venues_w_rate_df` for venues with ratings.  Preserving the core venues data and core ratings as their own data entities.<jupyter_code># Merge in ratings from stored ratings DataFrame... to clean the across multiple runs
venues_w_rate_df =  pd.merge(venues_df, ratings_df.drop_duplicates(subset=['Venue Id']), on=['Venue Id', 'Venue'], how='left')
print (venues_w_rate_df.shape)
venues_w_rate_df.head()<jupyter_output>(1037, 11)<jupyter_text>Transform venue list Transform the venue list so that it is grouped at "per location" basis<jupyter_code># One Hot Encode the venues based on Top Parent Category ...
# one hot encoding
venues_onehot = pd.get_dummies(venues_w_rate_df[['Venue Root Category']], prefix="", prefix_sep="")

# add Event Location column back to dataframe
venues_onehot['Event Location'] = venues_w_rate_df['Event Location'] 

# move neighborhood column to the first column
fixed_columns = [venues_onehot.columns[-1]] + list(venues_onehot.columns[:-1])
venues_onehot = venues_onehot[fixed_columns]

venues_onehot.head()
venues_onehot.shape
# group and find total number of each category
venues_grouped = venues_onehot.groupby('Event Location').sum().reset_index()
print (venues_grouped.shape)
venues_grouped<jupyter_output>(23, 9)<jupyter_text>Save the Summarized Venue Data<jupyter_code>## Save Event Lcoation Venue data
## BE CAREFUL, Overwrite = TRUE

project.save_data("event_location_venue_summary.csv", venues_grouped.to_csv(index=False), overwrite=True)<jupyter_output><empty_output><jupyter_text>Load the Data from the CSV to pick up where we left off<jupyter_code>venues_grouped = pd.read_csv(project.get_file("event_location_venue_summary.csv"))
print (venues_grouped.shape)
venues_grouped.head()<jupyter_output>(23, 9)<jupyter_text>---  ---    Exploratory Data Analysis   The different aspects of analysis are in no particular order    Explore Venue Proximity & Venue Rating around locations  Below are histograms summarzing     1. The Venue's Distance from event location    1. The Venue's Rating distribution    **Observations:**  The further you get from an event location, the more options there tends to be.  This is not surprising since the data set is list of high schools, and most high school campuses do not have many external venues operating on the campus.  I would expect this pattern to be true of many event locations.  With the limited access to the premium Foursquare API, most venue's do not have any rating data.  At this time, due to the large amount of missing data, no analysis will incorporate rating data.<jupyter_code># Plot distribution of Venue Ratings as well as Geodesic Distance from Sporting Event Location
fig = plt.figure(figsize=(10,5))
ax = fig.gca()
venues_df[['rating', 'Geodesic Distance From Event']].hist(ax=ax)
buffer = io.BytesIO()
#fig.show()
plt.savefig(buffer, format='jpg')
buffer.seek(0)
project.save_data('venue_radius_and_ratings.png', buffer, overwrite=True)<jupyter_output>/opt/conda/envs/Python36/lib/python3.6/site-packages/IPython/core/interactiveshell.py:3267: UserWarning: To output multiple subplots, the figure containing the passed axes is being cleared
  exec(code_obj, self.user_global_ns, self.user_ns)<jupyter_text>Home location to Away locations:  Travel & High Level Venue Analysis- Plot how far each away game is (distance)?  - how long will it take to get there (duration) ? - genenerally what's around each school ?The first analysis is to get a feel for the total population data.  So all schools on the schedule, not just the away games  **Note**  Since athlete schedules tend to remain fairly stable in terms of the same opponents every year, but each year they change locations.  So a game location "A" might be home this year will be away next year.  Thus having a general feel all schools is useful.**Observations**  The Distance and Duration plots seem like they will cluster relatively smoothly.  In the "all data" plots, no driving or duration bin contains more than 5 schools.  For the "Nearby Venue" histograms, there is a right skew, which might cause some challenges to clustering.<jupyter_code>school_display_cols = ['home_away', 'date', 'time', 'opponent_school', 'distance_miles', 'duration_minutes', 'duration_str', 'Nearby Venue Count']
# Driving Analysis to all schools 
# Plot how far each school is, how long will it take to get there, genenerally what's around each school ?

q_col_list = ['distance_miles', 'duration_minutes']
fig = plt.figure(figsize=(10,4))
ax = fig.gca()
games_loc_df[q_col_list].hist(ax=ax)
fig.suptitle('Count of Schools at a Distance or Duration; All Schools', fontsize=14)
plt.subplots_adjust(top=0.80)



buffer = io.BytesIO()
plt.savefig(buffer, format='png')
buffer.seek(0)
project.save_data('all_event_location_driving_hist_1.png', buffer, overwrite=True)

# Venue Analysis to all schools 
# genenerally what's around each school ?

q_col_list = ['Nearby Venue Count']
fig = plt.figure(figsize=(6,4))
ax = fig.gca()
games_loc_df[q_col_list].hist(ax=ax)
fig.suptitle('Count of Schools with a number of venues; All Schools', fontsize=14)
plt.subplots_adjust(top=0.80)


buffer = io.BytesIO()
plt.savefig(buffer, format='png')
buffer.seek(0)
project.save_data('all_event_location_venues_hist_1.png', buffer, overwrite=True)<jupyter_output><empty_output><jupyter_text>Now Repeat this analysis only for the away games<jupyter_code># Create a DataFrame of just the Away Games
away_games_df = games_loc_df.loc[games_loc_df['home_away'] == 'Away'] 

# Distance Analysis
#
# Plot how far each away game is, how long will it take to get there, genenerally what's around each school ?
fig = plt.figure(figsize=(10,4))
ax = fig.gca()
q_col_list = ['distance_miles', 'duration_minutes']
school_display_cols = ['home_away', 'date', 'time', 'opponent_school', 'distance_miles', 'duration_minutes', 'duration_str', 'Nearby Venue Count']
away_games_df[q_col_list].hist(ax=ax)
fig.suptitle('Count of Schools at a Distance or Duration; Away Locations', fontsize=14)
plt.subplots_adjust(top=0.80)

buffer = io.BytesIO()
plt.savefig(buffer, format='png')
buffer.seek(0)
project.save_data('away_location_driving_hist_1.png', buffer, overwrite=True)

# Venue Analysis
#
# Plot how far each away game is, how long will it take to get there, genenerally what's around each school ?
fig = plt.figure(figsize=(6,4))
fig.suptitle('Away Game Venue')
ax = fig.gca()
q_col_list = ['Nearby Venue Count']
school_display_cols = ['home_away', 'date', 'time', 'opponent_school', 'distance_miles', 'duration_minutes', 'duration_str', 'Nearby Venue Count']
away_games_df[q_col_list].hist(ax=ax)
fig.suptitle('Count of Schools with a number of venues; Away Locations', fontsize=14)
plt.subplots_adjust(top=0.80)

buffer = io.BytesIO()
plt.savefig(buffer, format='png')
buffer.seek(0)
project.save_data('away_location_venue_hist_1.png', buffer, overwrite=True)<jupyter_output><empty_output><jupyter_text>Numerical Summary of Locations<jupyter_code># Get numerical summary of All locations
games_loc_df.describe()
# Numerical Summary of Away Games
away_games_df.describe()<jupyter_output><empty_output><jupyter_text>Exploring Clustering  One of the goals is to assign labels to location to signal that additional planning is required for this location due to limited number of venues. Based on the histograms above lets try simple binning for initial analysis.  **NOTES**    - Since the actual travel to a location is more important than distance, we'll start with binning by `duration_minutes`.    - To simplify the analysis, we'll focus on `Food` and `Shop & Services` top-level categories, since the analysis is based on travelling for high-school basketball games.   First we'll merge the location data with venue data.<jupyter_code># Create a merged DataFrame
# Location + Venue Merged ... and only save Food + Shop & Services 
loc_ven_merged_df = pd.merge(games_loc_df, venues_grouped[['Event Location','Food', 'Shop & Service']], how='left', 
                             left_on='opponent_school', right_on='Event Location')
print(loc_ven_merged_df.shape)
loc_ven_merged_df.head()<jupyter_output>(23, 18)<jupyter_text>Bin based on duration  Let's try binning on duration minutesAfter the binning do some simple analysis to check the distrubtion of labels.  3 seems like a good number to start with.<jupyter_code># Try bin'ing

bin_num = 3
group_names = ['Short', 'Medium', 'Long']
loc_ven_merged_df['drive_label'] = pd.cut(loc_ven_merged_df['duration_minutes'], bin_num, labels=group_names, include_lowest=True)
# How many locations for each label 
print (loc_ven_merged_df['drive_label'].value_counts())

# Range captured in each label

for l in ['Short', 'Medium', 'Long']:
    print('\n')

    label_min = loc_ven_merged_df.loc[loc_ven_merged_df['drive_label'] == l][['duration_minutes']].nsmallest(1, 'duration_minutes').min()
    label_max = loc_ven_merged_df.loc[loc_ven_merged_df['drive_label'] == l][['duration_minutes']].nlargest(1, 'duration_minutes').max()
    print('Min {} Duration'.format(l.upper()), label_min)
    print('Max {} Duration'.format(l.upper()), label_max)
    print('{} Label Range'.format(l.upper()), label_max - label_min)<jupyter_output>Medium    12
Long       7
Short      4
Name: drive_label, dtype: int64


Min SHORT Duration duration_minutes    12.981667
dtype: float64
Max SHORT Duration duration_minutes    50.851667
dtype: float64
SHORT Label Range duration_minutes    37.87
dtype: float64


Min MEDIUM Duration duration_minutes    55.226667
dtype: float64
Max MEDIUM Duration duration_minutes    88.476667
dtype: float64
MEDIUM Label Range duration_minutes    33.25
dtype: float64


Min LONG Duration duration_minutes    99.045
dtype: float64
Max LONG Duration duration_minutes    135.818333
dtype: float64
LONG Label Range duration_minutes    36.773333
dtype: float64<jupyter_text>Duration Binning Observations  The duration binning results look good.  Each label's range roughly 35 minutes, most of the data is `Medium` which reflects reality, and would help.  This short label for duration will convey useful information.     Binning  based on Food Options  Lets try binning based on food options.  Starting with 3.<jupyter_code>bin_num = 3
group_names = ['A Few', 'Some', 'Lots']
loc_ven_merged_df['food_options'] = pd.cut(loc_ven_merged_df['Food'], bin_num, labels=group_names, include_lowest=True)

# How many locations for each label 
print (loc_ven_merged_df['food_options'].value_counts())

# Range captured in each label

for l in group_names:
    print('\n')

    label_min = loc_ven_merged_df.loc[loc_ven_merged_df['food_options'] == l][['Food']].nsmallest(1, 'Food').min()
    label_max = loc_ven_merged_df.loc[loc_ven_merged_df['food_options'] == l][['Food']].nlargest(1, 'Food').max()
    print('Min {} Duration'.format(l.upper()), label_min)
    print('Max {} Duration'.format(l.upper()), label_max)
    print('{} Label Range'.format(l.upper()), label_max - label_min)<jupyter_output>A Few    12
Some      7
Lots      4
Name: food_options, dtype: int64


Min A FEW Duration Food    0
dtype: int64
Max A FEW Duration Food    18
dtype: int64
A FEW Label Range Food    18
dtype: int64


Min SOME Duration Food    20
dtype: int64
Max SOME Duration Food    34
dtype: int64
SOME Label Range Food    14
dtype: int64


Min LOTS Duration Food    49
dtype: int64
Max LOTS Duration Food    56
dtype: int64
LOTS Label Range Food    7
dtype: int64<jupyter_text>Observations  for 3 BinsThe right hand skewness of the data has made the label representing the fewest options ('A Few') very large (over 50% of the data), and the upper bound location in this label has 18 venues.    My intuitions says planning a trip for location with 0 food options is not the same thing as planning a trip to a location with 18 food option.  It is an order of magnitude difference (1 vs 10).    **Repeating the analysis with 4 buckets (bins)**<jupyter_code>bin_num = 4
group_names = ['Slim Pickins', 'A Few', 'Some', 'Lots']
loc_ven_merged_df['food_options'] = pd.cut(loc_ven_merged_df['Food'], bin_num, labels=group_names, include_lowest=True)

# How many locations for each label 
print (loc_ven_merged_df['food_options'].value_counts())

# Range captured in each label

for l in group_names:
    print('\n')

    label_min = loc_ven_merged_df.loc[loc_ven_merged_df['food_options'] == l][['Food']].nsmallest(1, 'Food').min()
    label_max = loc_ven_merged_df.loc[loc_ven_merged_df['food_options'] == l][['Food']].nlargest(1, 'Food').max()
    print('Min {} Duration'.format(l.upper()), label_min)
    print('Max {} Duration'.format(l.upper()), label_max)
    print('{} Label Range'.format(l.upper()), label_max - label_min)<jupyter_output>Slim Pickins    10
A Few            5
Lots             4
Some             4
Name: food_options, dtype: int64


Min SLIM PICKINS Duration Food    0
dtype: int64
Max SLIM PICKINS Duration Food    12
dtype: int64
SLIM PICKINS Label Range Food    12
dtype: int64


Min A FEW Duration Food    17
dtype: int64
Max A FEW Duration Food    28
dtype: int64
A FEW Label Range Food    11
dtype: int64


Min SOME Duration Food    30
dtype: int64
Max SOME Duration Food    34
dtype: int64
SOME Label Range Food    4
dtype: int64


Min LOTS Duration Food    49
dtype: int64
Max LOTS Duration Food    56
dtype: int64
LOTS Label Range Food    7
dtype: int64<jupyter_text>Observations for 4 Bins  Intuitively, these results look much more useful.  The distance between max venue count of `Slim Pickins` and the min venue count of `A Few` is 5.  The smallest label, while still the largest label, is now ~43% of the total data (a drop from 53% of total data for the 3-bin cut)4 Bins for Food options is good choice.  Lets Try K-Means  The above binning model solely focused on `Food` options.  Lets try K-Means clustering on all available features Using the just the venue summarization data.First step is to find a good `K`.<jupyter_code>from sklearn.cluster import KMeans

list_k = range(1, 8)
_grouped_clustering = venues_grouped.drop('Event Location', axis=1)

sse = []
for k in list_k:
    print('Running KMeans for k=', k)
    km = KMeans(n_clusters=k)
    km.fit(_grouped_clustering)
    sse.append(km.inertia_)

# Plot sse against k
plot_k = list(range(1,8))
plt.figure(figsize=(6, 6))
plt.plot(plot_k, sse, '-o')
plt.xlabel(r'Number of clusters *k*')
plt.ylabel('Sum of squared distance');<jupyter_output><empty_output><jupyter_text>Based on the **elbow method** for cluser size, it seems 3 is suggested answer.  Lets Run KMeans, for `k=3`, and compare those labels to binning with 4 buckets.  And for comparison to binning, lets also run `k=4`<jupyter_code># use K = 3
k_clusters = 3
kmeans = KMeans(n_clusters=k_clusters, random_state=0).fit(_grouped_clustering)

# check cluster labels generated for each row in the dataframe
k3_labels = kmeans.labels_ 
k3_centers = kmeans.cluster_centers_
# Lets also use K = 4
k_clusters = 4
kmeans = KMeans(n_clusters=k_clusters, random_state=0).fit(_grouped_clustering)

# check cluster labels generated for each row in the dataframe
k4_labels = kmeans.labels_ 
k4_centers = kmeans.cluster_centers_
# Lets add the labels back to 
venues_grouped.insert(0,'K3 Labels' , k3_labels)
venues_grouped.insert(0,'K4 Labels' , k4_labels)

print(venues_grouped['K3 Labels'].value_counts())
print(venues_grouped['K4 Labels'].value_counts())
venues_grouped.groupby(['K3 Labels']).agg({'Food': ['sum', 'mean', 'count', 'min', 'max']})<jupyter_output><empty_output><jupyter_text>**Observation for K-MEans(K=3)**  K-Means with K=3, has the same issue as binning with 3 buckets.  The smallest bucket contains too much data to be practically useful<jupyter_code>venues_grouped.groupby(['K4 Labels']).agg({'Food': ['sum', 'mean', 'count', 'min', 'max']})<jupyter_output><empty_output><jupyter_text>**Observation for K-Means(K=4)**  The clusters are similarly sized (good thing)The cluster for locations with least number of venues are all in same order of magnitude (i.e. < 10) ...(good thing) Ranges are 9, 7, 9, 12 ... so clusters are dense (good thing) ---  ---   Model SelectionDo a quick model evaluation (since it is not easy to pass in the binning model to classic model scoring, use a custom methodology)  - Weight Average, determined by the range of values per location in a cluster * items in that cluster.      For example, if location A, with 16 nearby venues is in the same cluster as Location B, with 22 nearby venues, the range is 6. (i.e. 22-16 = 6).  If there 7 items in the cluster, then the "weight" of that cluster is 7 * 6 = 42            - The difference between the minimum cluster size and maximum cluster size.  To measure the 'balance' the model has in assigning labels<jupyter_code># Used a weighted average of size of cluster * items in cluster)
print ('kmeans(k=4)', (9 * 5 + 7*4 + 9*8 + 12*6)/4)
print ('kmeans(k=3)', (18 * 12 + 7*4 +  14*7)/3)
print ('Binning(bins=3)', (12*18 + 7*14 + 7*4)/3)
print ('Binning(bins=4)', (10*12 + 5*11 + 4*4 + 4*7)/4)

# Check Max - Min bucket delta, for items in each cluster
print ('kmeans(k=4)', 8 - 4)
print ('kmeans(k=3)', 12 - 4)
print ('Binning(bins=3)', 12 - 4)
print ('Binning(bins=4)', 10 - 4)<jupyter_output>kmeans(k=4) 54.25
kmeans(k=3) 114.0
Binning(bins=3) 114.0
Binning(bins=4) 54.75
kmeans(k=4) 4
kmeans(k=3) 8
Binning(bins=3) 8
Binning(bins=4) 6<jupyter_text>KMeans(K=4) is best modelKMeans produced the smallest average cluster weight, as well as the most consistent cluster sizes.<jupyter_code># Use the grouping analysis above, to match KMeans assigned groupings to the original labels from binning
# Then after udpate is done, re-run the groupby statemetns to see a more human readable version
# Lets change labels to match binning
venues_grouped['K3 Labels'].replace({0: 'Some', 1: 'A Few', 2:'Lots' }, inplace=True)
venues_grouped['K4 Labels'].replace({0: 'Some', 1: 'Slim Pickins', 2:'Lots', 3:'A Few'  }, inplace=True)<jupyter_output><empty_output><jupyter_text>Lets compare the multiple models by looking at actual dataBinning & Kmeans<jupyter_code>multi_model_df = pd.merge(loc_ven_merged_df[['opponent_school', 'home_away', 'distance_miles', 'duration_minutes', 'Food', 'Nearby Venue Count', 'drive_label', 'food_options']],
                          venues_grouped[['Event Location', 'K3 Labels', 'K4 Labels']], left_on='opponent_school', right_on='Event Location')
multi_model_df.drop(['Event Location'], axis=1, inplace=True)
multi_model_df.rename({'food_options': 'Bin Labels'}, axis=1, inplace=True)
print(multi_model_df.shape)
multi_model_df<jupyter_output>(23, 10)<jupyter_text>**Observations**  Comparing Binning(buckets=4) to Kmeans(k=4):  2 schools left `Slim Pickins` category in Binning model to `A Few` in KMeans model2 schools left `A Few` category in Binning model to `Some` in KMeans model.Below is a snapshot of one such school.<jupyter_code>#  Lawrence Academy is one of the schools that moved from `Slim Pickins` to `A Few` , Looking at data to see if that "Makes Sense"

venues_w_rate_df.loc[(venues_w_rate_df['Event Location'] == 'Lawrence Academy') & (venues_w_rate_df['Venue Root Category'] == 'Food')]
# Lets Save the multi-model DataFrame

project.save_data("multi_model_data.csv", multi_model_df.to_csv(index=False), overwrite=True)<jupyter_output><empty_output><jupyter_text>---  ---   Reporting & Presentation  Support This section is miscellaneous data "slicing & dicing", as well as additional visualizations to add the final presentation to coaches to make for a more compelling story.<jupyter_code>multi_model_df['food_options'] = multi_model_df['K4 Labels']
from pandas.plotting import table

table_df = multi_model_df.loc[multi_model_df['home_away']=='Away'][['opponent_school', 'home_away', 'distance_miles', 'duration_minutes', 'drive_label', 'food_options']]
table_df.set_index('opponent_school', inplace=True)

fig = plt.figure(figsize=(12,6))
ax = plt.subplot(111, frame_on=False )
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)

tbl = table(ax, np.round(table_df), loc='center' )
tbl.auto_set_font_size(False)
tbl.set_fontsize(14)
tbl.scale(1.5, 1.5)

fig.suptitle('Table of Away Games', fontsize=14)
plt.subplots_adjust(top=0.96)

buffer = io.BytesIO()
plt.savefig(buffer, format='png', bbox_inches='tight')
buffer.seek(0)
project.save_data('table_away_games.png', buffer, overwrite=True)



venues_grouped['Food'].describe()
venues_grouped.head()<jupyter_output><empty_output><jupyter_text>Just Play with the data  little bit ...Not yet for inclusion in the final report or presentation, just randomly play with the data see if anything interesting pops out...<jupyter_code>games_loc_df.loc[games_loc_df['distance_miles'] < 25]
long_trip_duration = 75  # Define 75 minutes as a "Long Trip", since duration is only one-way.  This is a 2hr 30 minute total driving time  
games_loc_df.loc[(games_loc_df['duration_minutes'] > long_trip_duration) & (games_loc_df['home_away'] == 'Away')][school_display_cols]
# 3 Closest Trips
_cols = ['opponent_school', 'date', 'distance_miles', 'duration_minutes', 'duration_str']
table_df = games_loc_df.loc[games_loc_df['home_away'] == 'Away'][_cols].nsmallest(3, 'duration_minutes')
table_df.set_index('opponent_school', inplace=True)

fig = plt.figure(figsize=(10,3))
ax = plt.subplot(111, frame_on=False )
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)

tbl = table(ax, np.round(table_df), loc='center' )
tbl.auto_set_font_size(False)
tbl.set_fontsize(14)
tbl.scale(1.5, 1.5)

fig.suptitle('Three Closest Games', fontsize=14)
plt.subplots_adjust(top=0.96)

buffer = io.BytesIO()
plt.savefig(buffer, format='png', bbox_inches='tight')
buffer.seek(0)
project.save_data('three_closest_games.png', buffer, overwrite=True)



# 3 Furthest Trips

_cols = ['opponent_school', 'date', 'distance_miles', 'duration_minutes', 'duration_str']
table_df = games_loc_df.loc[games_loc_df['home_away'] == 'Away'][_cols].nlargest(3, 'duration_minutes')

table_df.set_index('opponent_school', inplace=True)

fig = plt.figure(figsize=(10,3))
ax = plt.subplot(111, frame_on=False )
ax.xaxis.set_visible(False)
ax.yaxis.set_visible(False)

tbl = table(ax, np.round(table_df), loc='center' )
tbl.auto_set_font_size(False)
tbl.set_fontsize(14)
tbl.scale(1.5, 1.5)

fig.suptitle('Three Farthest Games', fontsize=14)
plt.subplots_adjust(top=0.96)

buffer = io.BytesIO()
plt.savefig(buffer, format='png', bbox_inches='tight')
buffer.seek(0)
project.save_data('three_farthest_games.png', buffer, overwrite=True)



# Find the 3 closest Food Venues for each away game 
rpt_df = venues_w_rate_df.loc[venues_w_rate_df['Venue Root Category'] == 'Food'].rename({'Geodesic Distance From Event': 'crow_distance'}, axis=1)
#rpt_df = rpt_df.loc(rpt_df['Venue Root Category'] == 'Food']
#rpt_df.set_index('Event Location')
_cols = ['Venue', 'Venue Category', 'crow_distance']
for _s in away_games_df['opponent_school']:
    print (_s)
    print (np.round(rpt_df.loc[rpt_df['Event Location'] == _s][_cols], 2).nsmallest(3, 'crow_distance').to_string(index=False))
    print ('\n\n')

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
print(scaler.fit(test_df['sub_total'].to_numpy().reshape(-1, 1)))
test_df['location_score'] = scaler.transform(test_df['sub_total'].to_numpy().reshape(-1, 1))

test_df.nsmallest(3, 'location_score')
test_df.nlargest(3, 'location_score')
col_list = school_display_cols + ['food_options', 'drive_label', 'Food']
test_df.loc[test_df['drive_label'] == 'Medium'][col_list]
print(test_df.loc[games_loc_df['home_away'] == 'Away']['dur-bin'].value_counts())
print(test_df.loc[games_loc_df['home_away'] == 'Away']['Loc-opts'].value_counts())
x = test_df.loc[games_loc_df['home_away'] == 'Away']['duration_minutes']
y = test_df.loc[games_loc_df['home_away'] == 'Away']['Nearby Venue Count']
area = y
plt.scatter(x, y, s=area,  alpha=0.5)
plt.show()
merged_test_df.loc[merged_test_df['home_away'] == 'Away'][['opponent_school','dur-bin','Loc-opts', 'Nearby Venue Count', 'Food', 'Shop & Service']]

merged_test_df = pd.merge(test_df, venues_grouped[['Event Location','Food', 'Shop & Service']], how='left', 
         left_on='opponent_school', right_on='Event Location')
merged_test_df.head()

# Come up with a School Location Score ... 
# what % of venues are Food & Shops 


venues_df.loc[(venues_df['Event Location'] == 'Holderness School') & (venues_df['Venue Root Category'] == 'Food' )]

import seaborn as sns
sns.set(font_scale=1.2)
plt.figure(figsize=(14,16))
ax = plt.axes()
sns.heatmap(venues_grouped.set_index('Event Location'), linewidths=1., cmap='YlGnBu', annot=True, fmt="d", ax=ax)
ax.set_title("Venue Count Per School by Category")

buffer = io.BytesIO()
plt.savefig(buffer, format='png', bbox_inches='tight')
buffer.seek(0)
project.save_data('venue_school_heatmap.png', buffer, overwrite=True)

venues_grouped[['Event Location','Food', 'Shop & Service']].head()
venues_grouped.head()<jupyter_output><empty_output><jupyter_text>---  ---<jupyter_code>games_loc_df.columns
away_games_df = games_loc_df.loc[games_loc_df['home_away'] == 'Away']

home_base_map = folium.Map(location=[42.63286695, -71.9135057096491], zoom_start=8)
folium.Marker(location=[42.63286695, -71.9135057096491], 
              popup='Cushing Academy',
             icon=folium.Icon(icon='home', color='purple', prefix='fa')).add_to(home_base_map)
# add markers to map
for lat, lng, opponent_school, _date in zip(away_games_df['lat'], away_games_df['lng'], away_games_df['opponent_school'], away_games_df['date']):
    label = '{}; {}'.format( opponent_school, _date)
    label = folium.Popup(label, parse_html=True)
    folium.CircleMarker(
        [lat, lng],
        radius=5,
        popup=label,
        color='blue',
        fill=True,
        fill_color='#3186cc',
        fill_opacity=0.7, 
        parse_html=False).add_to(home_base_map)  
home_base_map<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11855670103092783, 'fraction_numerical': 0.021200208592402424, 'mean_word_length': 4.791497270298525, 'pattern_counts': {'":': 0, '<': 140, '<?xml version=': 0, '>': 144, 'https://': 5, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 2, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '38', 'dir': 'jupyter-structured-clean-dedup', 'id': '88478', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 17095, 'n_tokens_neox': 15865, 'n_words': 4876, 'path': 'IBM_CRS9_W2_Capstone.ipynb', 'repo_name': 'mcmasty/Coursera_Capstone', 'size': '49863'}