<jupyter_start><jupyter_text>![Callysto.ca Banner](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-top.jpg?raw=true)<jupyter_code>import pandas as pd
import requests<jupyter_output><empty_output><jupyter_text>Exploring an API with PythonThis notebook uses the python requests module to experiment with the (undocumented) NHL statistics API. This API is by the NHL to drive lots of the functionality on nhl.com such as the live game tracker. Here, we use it to document the process of exploring a new API in the notebook environment. All of the code that follows is rough and ready. It is fit for purpose, but shouldn't be considerd production ready. NHL APIThe NHL publish statistics on games, players, standings and lots of other things in an unpublished api at https://statsapi.web.nhl.com. Some enterprising people have started [documenting the various end points](https://github.com/dword4/nhlapi) but it is still very much a work in progress. Some specific examples of endpoints are e.g * GET https://statsapi.web.nhl.com/api/v1/game/ID/feed/live Returns all data about a specified game id including play data with on-ice coordinates and post-game details like first, second and third stars and any details about shootouts. The data returned is simply too large at often over 30k lines and is best explored with a JSON viewer. * GET https://statsapi.web.nhl.com/api/v1/game/ID/boxscore Returns far less detail than feed/live and is much more suitable for post-game details including goals, shots, PIMs, blocked, takeaways, giveaways and hits. * GET http://statsapi.web.nhl.com/api/v1/game/ID/content Complex endpoint returning multiple types of media relating to the game including videos of shots, goals and saves.  TeamsStart off by collecting a list of current teams. This is pretty static information so we could collect it other ways, but in principal as soon as the NHL decide to give a team to seattle we're ready to roll!<jupyter_code>response = requests.get("https://statsapi.web.nhl.com/api/v1/teams")
if response.status_code != 200:
    print("FAILED")
    response.raise_for_status()<jupyter_output><empty_output><jupyter_text>We examined the status_code for the response to make sure that it was successful (200 is HTTP for "OK"). This is a good idea, but we'll skip it in later cells for brevity.Since this API is undocumented, we will take a quick look to see what information is being returned from that endpoint. We can do this by asking for the request result as JSON (Javascript Object Notation)<jupyter_code>response.json().keys()
teams = response.json()['teams']
teams[0]<jupyter_output><empty_output><jupyter_text>We have the ordinary stuff like 'name' and 'location' along with some more detail. One thing to notice is the 'id' value, which is used by some of the other endpoints to uniquely identify a team. We can find a specific id by looping through the teams until we find the one we want.<jupyter_code>teamName = '<NAME>'

teams = response.json()['teams']
for team in teams:
    if team['name'] == '<NAME>':
        teamID = team['id']
        
print("teamID {}: {}".format(teamID, teamName))<jupyter_output>teamID 20: Calgary Flames<jupyter_text>Pandas is a very useful module which helps organize and manipulate data. The main abstraction used by pandas is the DataFrame. In it's simplest incarnation a DataFrame is simply a table of Data which can be filtered, combined, updated. Let's construct a dataframe with to look at the when the teams joined the league.<jupyter_code>teamDF = pd.DataFrame()

for team in teams:
    df = pd.DataFrame().from_records(
        {
        'firstYearOfPlay': int(team['firstYearOfPlay']),
        'name': team['name']
        }, index=[team['id']])

    if teamDF is None:
        teamDF = df
    else:
        teamDF = teamDF.append(df)

teamDF.sort_values('firstYearOfPlay')<jupyter_output><empty_output><jupyter_text>It looks like the data only covers the current franchise holders (or the Canadiens played a lot of really one sided games back in the early 1900s).  ScheduleThe schedule endpoint includes data on past, current and future games. First we will grab a list of games from 2017-2018 season up till yesterday.<jupyter_code>day = (pd.datetime.today() - pd.Timedelta("1 days"))
day.strftime('%Y-%m-%d')
params = {
    'startDate' : '2017-10-04',
    'endDate'   : (pd.datetime.today() - pd.Timedelta("1 days")).strftime('%Y-%m-%d'),
    'teamID'    : teamID
}
response = requests.get("https://statsapi.web.nhl.com/api/v1/schedule", params=params)

if response.status_code != 200:
    print("FAILED")
else:
    print(response.url)<jupyter_output>https://statsapi.web.nhl.com/api/v1/schedule?startDate=2017-10-04&endDate=2018-01-29&teamID=20<jupyter_text>The structure returned this time is a bit more complicated.<jupyter_code>response.json().keys()<jupyter_output><empty_output><jupyter_text>The 'dates' key contains most of the information. Each element inside it is a date with one or more games taking place, we will look at the first date (element "0") as an example.<jupyter_code>response.json()['dates'][0].keys()<jupyter_output><empty_output><jupyter_text>This time most of the good information is inside te 'games' key. Inside that, we have entries for the home and away teams. Remember from above, we set teamID to identify one team. Let's loop over all of the games in the range and find the games<jupyter_code>gameDays = response.json()['dates']

homeGames = []
awayGames = []
for day in gameDays:
    for game in day['games']:
        
        if game['teams']['home']['team']['id'] == teamID:
            homeGames.append(game['gamePk'])
        elif game['teams']['away']['team']['id'] == teamID:
            awayGames.append(game['gamePk'])

print("home/away: {0}/{1}".format(len(homeGames), len(awayGames)))<jupyter_output>home/away: 26/23<jupyter_text>GamesMost of the detail in the API is in the game endpoints. For example, let's take the last home game from above and pull out the boxscore<jupyter_code>response = requests.get(
    'https://statsapi.web.nhl.com/api/v1/game/{}/boxscore'.format(homeGames[-1])
)

if response.status_code != 200:
    print("FAILED")
else:
    print(response.url)
response.json().keys()<jupyter_output><empty_output><jupyter_text>We have everything from the officials to the birthplacesof each skater! Let's look at the shot totals as a function of time<jupyter_code>response.json()['teams']['home']['teamStats']<jupyter_output><empty_output><jupyter_text>OK, let's build a dataframe with the data above for each game<jupyter_code>pd.DataFrame.from_records(response.json()['teams']['home']['teamStats']['teamSkaterStats'],
                          index=[homeGames[-1]]
                         )<jupyter_output><empty_output><jupyter_text>**N.B. This cell will make lots of requests**Most API providers will rate limit requests at some level, but since the nhl stats API is largely undocumented, I don't know what they will be! The cell below will take all home games (to date) and ask for their box scores as quickly as it can. If you run into errors take a look at the contents of response (raise_for_status())<jupyter_code>homeGameDF = pd.DataFrame()

for game in homeGames:
    response = requests.get(
        'https://statsapi.web.nhl.com/api/v1/game/{}/boxscore'.format(game)
    )

    if response.status_code != 200:
        response.raise_for_status()
    else:
        response.json()['teams']['home']['teamStats']['teamSkaterStats']
        df = pd.DataFrame().from_records(
            response.json()['teams']['home']['teamStats']['teamSkaterStats'],
            index=[game]
        )
        
    if homeGameDF is None:
        homeGameDF = df
    else:
        homeGameDF = homeGameDF.append(df)

homeGameDF
homeGameDF.sort_index(inplace=True)
homeGameDF['shots'].plot(figsize=(12, 6), style=['b+-'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10743592980684256, 'fraction_numerical': 0.010352228254008332, 'mean_word_length': 4.4446735395189005, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 10, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '218447', 'license': 'CC-BY-4.0', 'n_tokens_mistral': 2443, 'n_tokens_neox': 2343, 'n_words': 839, 'path': 'notebooks/General_Interest/NHL-Requests.ipynb', 'repo_name': 'callysto/callysto-sample-notebooks', 'size': '7937'}