<jupyter_start><jupyter_text>Hypothesis Testing<jupyter_code>import pandas as pd
pd.set_option('display.max_columns', None)
import numpy as np
# package with hypothesis tests
import scipy.stats as st<jupyter_output><empty_output><jupyter_text>Data You can download the data from [**here**](https://drive.google.com/file/d/19b9lHlkixZhs8yka8zV0QFieao66dUcY/view?usp=sharing). The data contains results of NBA games from seasons 2013/2014 to 2015/2016.  TaskSplit the data into **3** separate files according to the season!<jupyter_code>nba_csv = pd.read_csv('nba_games_2013_2015.csv', sep=';')
nba_csv.head()
nba_csv['SEASON_ID'].unique()
nba_2013 = nba_csv[nba_csv['SEASON_ID']==22013].copy()
nba_2014 = nba_csv[nba_csv['SEASON_ID']==22014].copy()
nba_2015 = nba_csv[nba_csv['SEASON_ID']==22015].copy()
nba_2015.head()
CC_filter = (nba_2015['TEAM_NAME'] == 'Cleveland Cavaliers') 
GSW_filter = (nba_2015['TEAM_NAME'] == 'Golden State Warriors')
CC_df15 = nba_2015[CC_filter]
GSW_df15 = nba_2015[GSW_filter]
CC_df15.head()<jupyter_output><empty_output><jupyter_text>TaskTest the hypothesis that offensive productions of Cleveland Cavaliers and Golden State Warriors (teams that met in the finals that year) were distributed equally in 2015/2016.Offensive production consists of two variables: PTS (Points) and FG_PCT (Field Goal Percentage). We need to do two separate hypothesis tests, one for each variable.<jupyter_code># Null Hypothesis 1 PTS(CC) == PTS(GSW)
# Null Hypothesis 2 FG_PCT(CC) == FG_PCT(GSW)
# H1 PTS
CC_pts = CC_df15['PTS'].tolist()
GSW_pts = GSW_df15['PTS'].tolist()
#ttest_pts = st.stats.ttest_ind(CC_pts,GSW_pts)
pval_pts = st.stats.ttest_ind(CC_pts,GSW_pts)[1]
if pval_pts > 0.05:
    print('Accept Null Hypothesis of equivalency in PTS (Points)')
elif pval_pts < 0.05:
    print('Reject Null Hypothesis, PTS (Points) NOT equivalent')
# H2 FG_PCT
CC_fgpct = CC_df15['FG_PCT'].tolist()
GSW_fgpct = GSW_df15['FG_PCT'].tolist()
#ttest_pts = st.stats.ttest_ind(CC_pts,GSW_pts)
pval_fgpct = st.stats.ttest_ind(CC_fgpct,GSW_fgpct)[1]
if pval_fgpct > 0.05:
    print('Accept Null Hypothesis of equivalency in FG_PCT (Field Goal Percentage)')
elif pval_fgpct < 0.05:
    print('Reject Null Hypothesis, FG_PCT (Field Goal Percentage) NOT equivalent')<jupyter_output>Reject Null Hypothesis, FG_PCT (Field Goal Percentage) NOT equivalent<jupyter_text>TaskTest the hypothesis that the number of points (PTS) scored by Cleveland Cavaliers changed significantly after the coach change in the season 2015/2016. **Coach Blatt was fired on 24th of Jan, 2016**. Use the data from seasons 2014/2015 and 2015/2016 - those are years when Cleveland was coached by Blatt. We have two possible solutions here:- take the same amount of games from before and after and try t-test.- take all the games from before and after and look for the right test to compare two samples with different sizes<jupyter_code># Null Hypothesis (PTS beforeJan24,2016) > (PTS after Jan24,2016)
CC_filter = (nba_2014['TEAM_NAME'] == '<NAME>') 
CC_df14 = nba_2014[CC_filter]
pts_before1 = CC_df14['PTS'].tolist()

before_filter = CC_df15['GAME_DATE'] < '2016-01-24'
after_filter = CC_df15['GAME_DATE'] < '2016-01-24'
CC15_before = CC_df15[before_filter]
pts_before2 = CC15_before['PTS'].tolist()
CC15_after = CC_df15[after_filter]

pts_after = CC15_after['PTS'].tolist()
pts_before = pts_before1 + pts_before2

pval_pts = st.stats.ttest_ind(pts_before,pts_after)[1]
if pval_pts > 0.05:
    print('Accept Null Hypothesis: PTS (Points) before Jan 24, 2016 HIGHER with Coach Blatt')
elif pval_pts < 0.05:
    print('Reject Null Hypothesis: PTS (Points) did not change after firing Coach Blatt')<jupyter_output>Accept Null Hypothesis: PTS (Points) before Jan 24, 2016 HIGHER with Coach Blatt<jupyter_text>TaskDownload [**the similar dataset**](https://drive.google.com/file/d/1jY57bAOZp9y83b4W2PAoSH1uFARaxxls/view?usp=sharing) with scores from playoff games in 2016.<jupyter_code>playoffs16 = pd.read_csv('nba_games_2013_2015.csv', sep=';')
playoffs16.head()<jupyter_output><empty_output><jupyter_text>TaskTest the hypothesis that number of blocks (BLK) are from the same distribution in both, in playoffs and in the reguar season 2015/2016 for **Toronto Raptors**. We need to work with two samples with different sizes again.<jupyter_code>sn_filter = nba_2015['TEAM_NAME'] == 'Toronto Raptors'
po_filter = playoffs16['TEAM_NAME'] == 'Toronto Raptors'
blk_ssn = nba_2015[sn_filter]['BLK'].tolist()
blk_poff = playoffs16[po_filter]['BLK'].tolist()

# Null Hypothesis: Number of BLK (blocks) in season == playoffs for Toronto Raptors
pval_raptors = st.stats.ttest_ind(blk_ssn,blk_poff)[1]
if pval_raptors > 0.05:
    print('Accept Null Hypothesis: Number of BLK (blocks) in season == playoffs for Toronto Raptors')
elif pval_raptors < 0.05:
    print('Reject Null Hypothesis: Number of BLK (blocks) in season =/= playoffs for Toronto Raptors')<jupyter_output>Reject Null Hypothesis: Number of BLK (blocks) in season =/= playoffs for Toronto Raptors<jupyter_text>TaskTest the hypothesis that the number of points (PTS) scored by Cleveland Cavaliers is equally distributed for all 3 seasons. In this case, we need a hypothesis test that compares more than 2 distributions at the same.<jupyter_code>CC_13 = nba_2013[nba_2013['TEAM_NAME'] == 'Cleveland Cavaliers']['PTS'].tolist()
CC_14 = nba_2014[nba_2014['TEAM_NAME'] == 'Cleveland Cavaliers']['PTS'].tolist()
CC_15 = nba_2015[nba_2015['TEAM_NAME'] == 'Cleveland Cavaliers']['PTS'].tolist()
# Null Hypothesis: All PTS (Points) in all 3 seasons are equally distributed
pval_CC = st.stats.f_oneway(CC_13, CC_14, CC_15)[1]
if pval_CC > 0.05:
    print('Accept Null Hypothesis: PTS (Points) in all 3 seasons are equally distributed')
elif pval_CC < 0.05:
    print('Reject Null Hypothesis: PTS (Points) in all 3 seasons are NOT equally distributed')<jupyter_output>Reject Null Hypothesis: PTS (Points) in all 3 seasons are NOT equally distributed<jupyter_text>Follow UpBetween which seasons can we see the significant difference?+ unfortunatelly, this is not the output of the ANOVA test and further tests need to be run.+ note that <NAME> came back to Cleveland prior to season 2014/2015. We can use this fact to interpret the results correctly.<jupyter_code># testing 13/14 to 14/15
pval_13thru15 = st.stats.ttest_ind(CC_13,CC_14)[1]
if pval_13thru15 > 0.05:
    print('Accept Null Hypothesis: PTS (Points) between 13/14 and 14/15 seasons are equally distributed')
elif pval_13thru15 < 0.05:
    print('Reject Null Hypothesis: PTS (Points) between 13/14 and 14/15 seasons are NOT equally distributed-- <NAME> saved the team from desperation')
# testing 14/15 to 15/16
pval_14thru16 = st.stats.ttest_ind(CC_14, CC_15)[1]
if pval_14thru16 > 0.05:
    print('Accept Null Hypothesis: PTS (Points) between 14/15 and 15/16 seasons are equally distributed')
elif pval_14thru16 < 0.05:
    print('Reject Null Hypothesis: PTS (Points) between 14/15 and 15/16 seasons are NOT equally distributed')<jupyter_output>Accept Null Hypothesis: PTS (Points) between 14/15 and 15/16 seasons are equally distributed
{'fraction_non_alphanumeric': 0.10634407089604726, 'fraction_numerical': 0.06189337459558306, 'mean_word_length': 6.060575968222443, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 39, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 2, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '18721', 'license': 'MIT', 'n_tokens_mistral': 2856, 'n_tokens_neox': 2486, 'n_words': 739, 'path': 'Probability and Statistics/Hypothesis Testing II.ipynb', 'repo_name': 'robynmundle/data_science', 'size': '7134'}