<jupyter_start><jupyter_text>![Callysto.ca Banner](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-top.jpg?raw=true)Sports Analytics in the ClassroomTina Leard (Cybera) <NAME> (University of Calgary)  October 2020.This is a slideshow in Jupyter notebook format.To start, click on the "Run All" arrows above, or select the "Kernel" menu item and choose "Restart & run all." Say yes to the confirmation.To get to slideshow mode, click on the icon in the menu bar that looks like a bar chart, as shown here. <img src="images/slideshow_icon.png"     alt="Slideshow icon"     width=200 />     You can advance the slide show using the arrow keys. Exit by clicking on that big "X" in the top left corner of the slide show. Note the slide show is still a live Jupyter notebook, so you can run the code and edit while in the slide show.<jupyter_code># This only needs to be run once by the user. But I leave it in here for the first time users
!pip install ipysheet --user
## don't think I need this, so I commented it out
!jupyter nbextension enable --py --sys-prefix --user ipysheet<jupyter_output>Enabling notebook extension ipysheet/extension...
      - Validating: [32mOK[0m<jupyter_text>Getting started  --- if you don't have time for this whole presentation ---- Read about the Callysto resources at https://callysto.ca- Take an online course at https://courses.callysto.ca/- log into the Jupyter Hub at https://hub.callysto.ca- launch this talk yourself: https://tinyurl.com/Callysto2020  Outline- what is possible with sports analytics- what tools can we use in the classroom    - Python, Jupyter notebooks, data analysis- our examples with field hockey, soccer, baseball- dive into some code  What is possible with Sports Analytics<jupyter_code>%%html
<h1> Tracking soccer players for analysis </h1> <p> <iframe width="800" height="450" src="https://www.youtube.com/embed/N9qZ-I63BL8?start=552" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>What is possible with Sports Analytics<jupyter_code>%%html
<h1>Detailed visual analytics </h1><p> <iframe width="800" height="450" src="https://www.youtube.com/embed/_U_Q5jpeGMo?start=678" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>What is possible with Sports Analytics<jupyter_code>%%html
<h1>Detailed discussion on soccer analytics </h1> <p> <iframe width="800" height="450" src="https://www.youtube.com/embed/JW2QVXTnlbQ?start=1747" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe><jupyter_output><empty_output><jupyter_text>Some tools for the classroom  Python programming language- A powerful, easy to use, all-purpose computing language- Industry standard, taught in colleges and universities, and very popular- This slide show can run Python code<jupyter_code>2+2
for n in range(10):
    print(n,n*n)<jupyter_output>0 0
1 1
2 4
3 9
4 16
5 25
6 36
7 49
8 64
9 81<jupyter_text>Some tools for the classroom  Jupter notebooks. - This slide deck is a Jupyter notebook- It is a simple document that holds text, images and code- It is organized into a series of editable cells (show them)- It runs on a system called a Jupyter Hub, that can "live" in the cloud or on your PC  Some tools for the classroom  Jupter Hubs. - A hub is an environment to create, edit and "run" your Jupyter notebook- Like a word processor, PowerPoint presenter, and coding environment all in run- The Hub can run in the cloud (easy) or on your PC (requires a software install)- This slideshow can be run in the cloud with a simple mouse click:    - Binder [link](https://mybinder.org/v2/gh/callysto/Sports/main?filepath=SportsAnalytics.ipynb)    - callysto.ca [link]( https://hub.callysto.ca/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Fcallysto%2FSports&urlpath=tree%2FSports%2FSportsAnalytics.ipynb&branch=main)- makes easy to share your notebooks with students and colleagues  What we can do in Sports Analytics  Let's first look a some images  A Field Hockey Pitch<img src="images/field_hockey_pitch_h_0.png"     alt="Field Hockey Pitch"     width=800 />  Circle Zones in Goal AreaA background for visualization<img src="images/field_hockey_circle_zones.png"     alt="Field Hockey Circle Zones"     width=800 />  A Quidditch Pitch (for my son)<img src="images/quidditch_pitch.png"     alt="Quidditch Pitch"     width=800 /><jupyter_code>## import some code modules
from ipysheet import sheet, cell, cell_range

spread_sheet = sheet()
cell(1,1, "It's Alive!");<jupyter_output><empty_output><jupyter_text>What we can do  Using Spreadsheets, Live<jupyter_code>spread_sheet
# It seems to work better if I import this stuff after the ipysheet module abive
import plotly.graph_objects as go
import cufflinks as cf
cf.go_offline()
import pandas as pd
import numpy as np
%matplotlib inline
# We set up a spreadsheet, and a figure, that will be linked together.

# set these for number of lanes, number of zones
nLanes = 3
nZones = 4

# We need a list of Lanes and Zones. Could be long.
LaneList = ['Lane 1','Lane 2','Lane 3','Lane 4','Lane 5','Lane 6','Lane 7','Lane 8']
ZoneList = ['Zone 1','Zone 2','Zone 3','Zone 4','Zone 5','Zone 6','Zone 7','Zone 8']

# here is the spreadsheet, using ipysheets function

spread_sheet_1 = sheet(rows=nLanes, columns=nZones, \
                       column_headers=ZoneList[0:nZones], \
                       row_headers = LaneList[nLanes-1::-1])
my_cells = cell_range([[1,1,1,3],[2,2,1,3],[3,3,1,3]],row_start=0,col_start=0,numeric_format="int")


figure_1 = go.Figure(data=go.Heatmap(
          y = LaneList[0:nLanes],
          x = ZoneList[0:nZones],
          z =list(reversed(my_cells.value)),
          type = 'heatmap',
          colorscale = 'peach',opacity = 1.0))

axis_template = dict(range = [0,5], autorange = True,
             showgrid = False, zeroline = True,
             linecolor = 'black', showticklabels = True,
             ticks = 'outside' )

figure_1.update_layout(margin = dict(t=50,r=200,b=200,l=200),
    xaxis = axis_template,
    yaxis = axis_template,
    showlegend = False,
    width = 906, height = 556, title="Field Hockey Zones and Lanes",
    autosize = True )

# Add image in the background
figure_1.add_layout_image(
        dict(
            source="images/field_hockey_pitch_h_0.png",
            xref="x",
            yref="y",
            x=-0.5,
            y=-.5 + nLanes,  #this adjusts the placement of the field image
            sizex=nZones,
            sizey=nLanes,
            sizing="fill",
            opacity=0.7,
            layer="above")
)

# changes in my_cells should trigger this function
def calculate(change):
    figure_1.update_traces(z=list(reversed(my_cells.value)))
    
my_cells.observe(calculate, 'value')

figure_1.show()  # use this for debugging only<jupyter_output><empty_output><jupyter_text>What we can do  Link spread sheets to the images<jupyter_code>spread_sheet_1  # Enter data in this spreadsheet
figure_1.update() # Click the keys "Shift-return" to update the figure
# REDO THIS IN THE VERTICAL MODE

# set these for number of lanes, number of zones
nLanes = 5
nZones = 4

# We need a list of Lanes and Zones. Could be long.
LaneList = ['Lane 1','Lane 2','Lane 3','Lane 4','Lane 5','Lane 6','Lane 7','Lane 8']
ZoneList = ['Zone 1','Zone 2','Zone 3','Zone 4','Zone 5','Zone 6','Zone 7','Zone 8']

# here is the spreadsheet, using ipysheets function

spread_sheet_2 = sheet(rows=nZones, columns=nLanes, \
                       column_headers=LaneList[0:nLanes], \
                       row_headers = ZoneList[nZones-1::-1])
my_cells_2 = cell_range([[1,1,1,3,1],[2,2,1,3,1],[3,3,1,3,1],[1,2,1,2,3]],row_start=0,col_start=0,numeric_format="int")


figure_2 = go.Figure(data=go.Heatmap(
          y = ZoneList[0:nZones],
          x = LaneList[0:nLanes],
          z = list(reversed(my_cells_2.value)),
          type = 'heatmap',
          colorscale = 'peach',opacity = 1.0))

axis_template = dict(range = [0,5], autorange = True,
             showgrid = False, zeroline = True,
             linecolor = 'black', showticklabels = True,
             ticks = 'outside' )

figure_2.update_layout(margin = dict(t=0,r=0,b=0,l=250),
    xaxis = axis_template,
    yaxis = axis_template,
    showlegend = False,
    width = 600, height = 500, title="Field Hockey Zones and Lanes",
    autosize = True )

# Add image in the background
figure_2.add_layout_image(
        dict(
            source="images/field_hockey_pitch.png",
            xref="x",
            yref="y",
            x=-0.5,
            y=-0.5 +nZones,  #this adjusts the placement of the field image
            sizex=5,
            sizey=4,
            sizing="fill",
            opacity=0.7,
            layer="above")
)

# changes in my_cells should trigger this function
def calculate_2(change):
    figure_2.update_traces(z=list(reversed(my_cells_2.value)))
    
my_cells_2.observe(calculate_2, 'value')

figure_2.show()  # use this for debugging only<jupyter_output><empty_output><jupyter_text>Let's make this vertical<jupyter_code>spread_sheet_2 # Enter data in this spreadsheet
figure_2.update() # Click the keys "Shift-return" to update the figure<jupyter_output><empty_output><jupyter_text>Let's load in some real data<jupyter_code>df_passes = pd.read_csv('data/field_hockey_home_passes.csv'); df_passes<jupyter_output><empty_output><jupyter_text>Where the data come from<img src="images/passes_team.jpeg"     alt="Passes from home team"     width=600 /><jupyter_code>myLayout = cf.Layout(
    height=800,
    width=400
)
## Look at passes in the first quarter
df_temp = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 1) ]; df_temp
df_temp.iplot(kind='heatmap', colorscale='spectral', layout=myLayout, x='x', y='y', z='Count' )
# Look at passes in the third quarter
df_temp = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 3) ]; df_temp
df_temp.iplot(kind='heatmap', colorscale='spectral', layout=myLayout, x='x', y='y', z='Count' )
df_temp1 = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 1) ];
df_temp2 = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 2) ]; 
df_temp3 = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 3) ]; 
df_temp4 = df_passes.loc[lambda df: (df['Phase of Play'] == 'attack') &(df['Quarter'] == 4) ]; 
from plotly import subplots
from plotly.offline import download_plotlyjs, plot,iplot
#import plotly.tools as tls
fig_all = subplots.make_subplots(rows=1, cols=4)
fig_1 = df_temp1.iplot(kind='heatmap', colorscale='spectral', title='Q1',layout=myLayout, x='x', y='y', z='Count' , asFigure=True)
fig_2 = df_temp2.iplot(kind='heatmap', colorscale='spectral', layout=myLayout, x='x', y='y', z='Count' , asFigure=True)
fig_3 = df_temp3.iplot(kind='heatmap', colorscale='spectral', layout=myLayout, x='x', y='y', z='Count' , asFigure=True)
fig_4 = df_temp4.iplot(kind='heatmap', colorscale='spectral', layout=myLayout, x='x', y='y', z='Count' , asFigure=True)
fig_all.append_trace(fig_1['data'][0], 1, 1)
fig_all.append_trace(fig_2['data'][0], 1, 2)
fig_all.append_trace(fig_3['data'][0], 1, 3)
fig_all.append_trace(fig_4['data'][0], 1, 4)
iplot(fig_all)<jupyter_output><empty_output><jupyter_text>What do we notice in these heatmaps?- Look at all those zeros -- no passes, or few passes near one goal- High passes near the other goal - A play/coach might ask deeper questions than I do!  Visual Analytics- Video recording of an entire game, analyzed for events- record times, position of players, actions such as passes and shots on goal- saved in a large spread sheet, 30+ Megs of dataWe use the example of a soccer game<jupyter_code>df1 = pd.read_csv('data/Sample_Game_1_RawEventsData.csv'); df1  ## Sample game data set<jupyter_output><empty_output><jupyter_text>A first attempt to visualize the data (not informative)<jupyter_code>df1.plot();  # Naive attempt to plot the data (not informative)<jupyter_output><empty_output><jupyter_text>A better visualization: locate events on the field<jupyter_code>df1.iplot(kind="scatter",x = "Start X", y = "Start Y", mode = "markers")<jupyter_output><empty_output><jupyter_text>Beginning analysis: location of passes by "Away" team, first period<jupyter_code>df1.loc[lambda df: (df['Team'] == 'Away') & (df['Period'] == 1) & (df['Type'] == 'PASS'), :] \
   .iplot(kind="scatter",x = "Start X", y = "Start Y", mode = "markers")<jupyter_output><empty_output><jupyter_text>Compare with location of passes by "Home" team, first period<jupyter_code>df1.loc[lambda df: (df['Team'] == 'Home') & (df['Period'] == 1) & (df['Type'] == 'PASS'), :] \
   .iplot(kind="scatter",x = "Start X", y = "Start Y", mode = "markers")<jupyter_output><empty_output><jupyter_text>Location of "Ball Out" by "Away" team, first period<jupyter_code>df1.loc[lambda df: (df['Team'] == 'Away') & (df['Period'] == 1) & (df['Type'] == 'BALL OUT'), :] \
   .iplot(kind="scatter",x = "Start X", y = "Start Y", mode = "markers")<jupyter_output><empty_output><jupyter_text>Compare with location of "Ball Out" by "Home" team, first period<jupyter_code>df1.loc[lambda df: (df['Team'] == 'Home') & (df['Period'] == 1) & (df['Type'] == 'BALL OUT'), :] \
   .iplot(kind="scatter",x = "Start X", y = "Start Y", mode = "markers")<jupyter_output><empty_output><jupyter_text>Other analyses we can easily do with this data- Compare performance of "Home" team to the "Away" team- Compare first period to second period (number of passes, number of interceptions, shots on goal)- Compare individual players (number of passes, number of shots on goal)  Example: Baseball Statistics- students may like to follow their own professional sports team- the move **Moneyball** was all about using data analysis to improve a team- **<NAME>!**- Many webpages post current and historical MLB statistics<img src="images/BB_stats.png"     alt="Baseball stats webpage"     width=800 />  Accessing webpage data from Jupyter- can read the documentation on each webpage (MLB.com, baseball-reference.con, etc)- **OR INSTEAD,** recognize that someone else may have already written code to do it Baseball API- Application Programming Interface- here is an article on the MLB API:  [https://medium.com/better-programming/using-the-mlb-stats-api-to-get-daily-data-88f48266656c](https://medium.com/better-programming/using-the-mlb-stats-api-to-get-daily-data-88f48266656c)<jupyter_code>pip install MLB-StatsAPI --user
import statsapi
rookie_hr_leaders = statsapi.league_leaders('homeRuns',season=2019, playerPool = 'rookies', limit = 5)
rookie_hr_leaders = rookie_hr_leaders.split('\n')
rookie_hr_leaders
standings = pd.DataFrame(statsapi.standings_data(104)[203]['teams'])
standings<jupyter_output><empty_output><jupyter_text>More examples, from this github repo:https://github.com/toddrob99/MLB-StatsAPI/wiki/Examples What is a github repo- an online resource where people store their publically available computer code- lots of it is for use in Jupyter notebooks and Python- often with very good documentation  Print the number of games won by the Oakland Athletics in 2018<jupyter_code>print('The A\'s won %s games in 2018.' 
      % sum(1 for x in statsapi.schedule(team=133,start_date='01/01/2018',end_date='12/31/2018') 
            if x.get('winning_team','')=='Oakland Athletics'))<jupyter_output>The A's won 111 games in 2018.<jupyter_text>My son's comment> The last one gets something wrong -- the A's didn't win 111 games in 2018, they won 97.  How did he know?> 108 wins is a rare feat, because that is winning 2/3-rds of the season. So 111 sounded wrong, because the A's weren't the best in their division. So he looked up the right number.> The A's had 0 wins in the post season, 97 wins in the regular season, and 14 wins in spring training. So there's the problem -- counting preseason games.   What else can you do with baseball stats- **Lots!**- See the movie **Moneyball**  Diving into the code and technology  Notes on the technology- Created on the **Callysto.ca** technology - Callysto is all free, all open source, aimed at teachers and students- Python for the programming language- Using Jupyter notebooks to put all the pieces together, including the slideshow- Software libraries include:    - Pandas for data analysis    - Ipysheet for spread sheets    - Plotly and Cufflinks for graphics    - Numpy for numerical routines  Getting started- Read about the Callysto resources at https://callysto.ca- Take an online course at https://courses.callysto.ca/- log into the Jupyter Hub at https://hub.callysto.ca  Using the Callysto HubDemo:- https://hub.callysto.ca- show how to log in- show a few files- markdown and code  Some basic PythonDemo:- simple arithmetic- variables- data structures (tuples, lists, dictionaries)- loops, conditional statements<jupyter_code># Write our code here<jupyter_output><empty_output><jupyter_text>Using PandasDemo:- importing the Pandas code module- creating a dataframe- some simple dataframe manipulations<jupyter_code># write our code here<jupyter_output><empty_output><jupyter_text>Plotly and CufflinksDemo:- importing the code modules- simple plots- plotting from a dataframe<jupyter_code># write our code here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12262433139865711, 'fraction_numerical': 0.02367133265050643, 'mean_word_length': 4.575824873096447, 'pattern_counts': {'":': 0, '<': 113, '<?xml version=': 0, '>': 116, 'https://': 17, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '23', 'dir': 'jupyter-structured-clean-dedup', 'id': '221189', 'license': 'CC0-1.0', 'n_tokens_mistral': 6092, 'n_tokens_neox': 5692, 'n_words': 1788, 'path': 'SportsAnalytics.ipynb', 'repo_name': 'callysto/Sports', 'size': '17588'}