<jupyter_start><jupyter_text>End of module projectPART 1 - Analysis of the dataUses the tennis matches results data between 2020 and 2022 from http://tennis-data.co.uk/alldata.php<jupyter_code># do the imports
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import r2_score, mean_squared_error
%matplotlib inline

# read data file
df = pd.read_csv('./2020-2022.csv')
df.head()
# drop the lines where there are no data in AvgW, AvgL, WRank or LRank
# we could use one of the existing odds (other than average) but given the limited number of lines dropped it's no issue
# for the ranks on the other hand we really need to have both players' rank (to compare) so we drop the line where one is missing
# hence dropping the lines for AvgW and AvgL is more simple too
numels = df.shape[0]
df = df.dropna(subset=['AvgW','AvgL','WRank','LRank'],axis=0)
print('Drop NA on AvgW, AvgL, WRank, LRank: ',numels-df.shape[0],' rows dropped')

# drop the lines where AvgW ou AvgL are dubious (more than 50% away from B365W / B365L)
# means there could be a mistake in either one (again, very limited number of rows)
numels = df.shape[0]
df = df.drop(df[df['AvgW']>1.5*df['B365W']].index,axis=0)
df = df.drop(df[df['AvgL']>1.5*df['B365L']].index,axis=0)
print('Drop dubious odds: ',numels-df.shape[0],' rows dropped')

df = df.reset_index(drop=True)

# compute delta rank (difference between winner and loser ranks)
df['DeltaRank'] = df['WRank']-df['LRank']<jupyter_output>Drop NA on AvgW, AvgL, WRank, LRank:  15  rows dropped
Drop dubious odds:  10  rows dropped<jupyter_text>0. How reliable are the odds?<jupyter_code>def getOddsResult(df,diff):
    res = diff.copy()
    i=0
    for d in diff:
        df2 = df[abs(df['AvgL']-df['AvgW'])>d]
        ok = df2[df2['AvgW']<df2['AvgL']]
        res[i] = ok.size/df2.size*100
        i=i+1
    return(res)
# odds are ok if winner's was lower than loser's
# check results for all data
diff = [0,0.1,0.2,0.3,0.4,0.5,1,2,3,4,5]
res = getOddsResult(df,diff)
# check results for ATP250 tournaments
res250 = getOddsResult(df[df['Series']=="ATP250"],diff)
# check results for ATP500 tournaments
res500 = getOddsResult(df[df['Series']=="ATP500"],diff)
# check results for Masters 1000 tournaments
res1000 = getOddsResult(df[df['Series']=="Masters 1000"],diff)
# check results for Grand Slam tournaments
resGS = getOddsResult(df[df['Series']=="Grand Slam"],diff)
# show the results: rate of good prediction rate as a function of difference between odds
plt.plot(diff,res,color = 'red',label='All')
plt.plot(diff,res250,color = 'blue',label='ATP 250')
plt.plot(diff,res500,color = 'green',label='ATP 500')
plt.plot(diff,res1000,color = 'purple',label='M 1000')
plt.plot(diff,resGS,color = 'orange',label='GS')
plt.xlabel("Delta between W and L odds")
plt.ylabel("% of good prediction")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Grand Slams stand clearly above the rest in terms of reliability of win prediction considering only the odds. This seems to be in accordance with classical expectations, as Grand Slams are the most important tournaments of the year and players really get ready for those.<jupyter_code># check repartition of matches depending on difference in odds
subdf = df[abs(df['AvgL']-df['AvgW'])<=5]
plt.hist(abs(subdf['AvgL']-subdf['AvgW']),bins=50,cumulative=True,density=True)
plt.show()<jupyter_output><empty_output><jupyter_text>Focus on Grand slams tournaments (by round)<jupyter_code>#  sdf -> GRand Slam tournaments
sdf = df[df['Series']=="Grand Slam"]

# check results for 1st round
resGS1 = getOddsResult(sdf[sdf['Round']=="1st Round"],diff)
# check results for 2nd Round
resGS2 = getOddsResult(sdf[sdf['Round']=="2nd Round"],diff)
# check results for 3rd Round
resGS3 = getOddsResult(sdf[sdf['Round']=="3rd Round"],diff)
# check results for 4th Round
resGS4 = getOddsResult(sdf[sdf['Round']=="4th Round"],diff)
# check results for Quarterfinals
resGS5 = getOddsResult(sdf[sdf['Round']=="Quarterfinals"],diff)
# check results for Semifinals
resGS6 = getOddsResult(sdf[sdf['Round']=="Semifinals"],diff)
# check results for The Final
resGS7 = getOddsResult(sdf[sdf['Round']=="The Final"],diff)
# plot the results
plt.plot(diff,resGS1,color = 'red',label='1st')
plt.plot(diff,resGS2,color = 'blue',label='2nd')
plt.plot(diff,resGS3,color = 'green',label='3rd')
plt.plot(diff,resGS4,color = 'purple',label='4th')
plt.plot(diff,resGS5,color = 'orange',label="Quarter")
plt.plot(diff,resGS6,color = 'black',label='Semi')
plt.plot(diff,resGS7,color = 'gray',label='Final')
plt.xlabel("Delta between W and L odds")
plt.ylabel("% of good prediction")
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>Interestingly, we can notice then reliability is better from quarterfinals onwards, and first round is globally the less reliable (which seems logical)  Grand Slams -> gains by match and by round ALL MATCHES -> Simulate the gains if one was betting always according to the odds for each round<jupyter_code>def getOddsGain(df,bet):
    gain = 0
    for i in df.index:
        if df['AvgW'][i]>df['AvgL'][i]:
            gain = gain+bet*(df['AvgW'][i]-1)
        else:
            gain = gain-bet
    return(gain,gain/df.shape[0])
# check results for all rounds
rounds = ["1st Round","2nd Round","3rd Round","4th Round","Quarterfinals","Semifinals","The Final"]
for round in rounds:
    gain,gainbymatch = getOddsGain(sdf[sdf['Round']==round],10)
    print("Gains in",round,": ",gain," - gain by match: ",gainbymatch)<jupyter_output>Gains in 1st Round :  -872.6000000000004  - gain by match:  -1.7211045364891526
Gains in 2nd Round :  -511.4000000000001  - gain by match:  -2.0293650793650797
Gains in 3rd Round :  -137.3  - gain by match:  -1.0811023622047244
Gains in 4th Round :  -55.60000000000001  - gain by match:  -0.8825396825396826
Gains in Quarterfinals :  -119.10000000000001  - gain by match:  -3.7218750000000003
Gains in Semifinals :  -110.7  - gain by match:  -6.91875
Gains in The Final :  -24.0  - gain by match:  -3.0<jupyter_text>SELECTED MATCHES -> Simulate the gains if one was betting always according to the odds for each roundhere selection is betting on top 10 players when their oppoentn is lower ranked<jupyter_code>thr = 10 # threshold for players ranking - 10 means we consider top 10 players
delta = 0 # delta is the ranking difference with the player's opponent - 0 means any player with higher ranking
series = "Grand Slam" # the type of tournament we consider

round = ["1st Round","2nd Round","3rd Round","4th Round","Quarterfinals","Semifinals","The Final"] 
gains = [0,0,0,0,0,0,0]
gainspermatch = [0,0,0,0,0,0,0]
roi = [0,0,0,0,0,0,0]
k=0
for r in round:
    subdf = df.loc[((df['WRank']<=thr) & (df['LRank']>df['WRank']+delta) & (df['Series']==series) & (df['Round']==r))]
    subdf = subdf.reset_index(drop=True)

    gain = 0
    nmatch = subdf.shape[0]
    print("win:",nmatch," average odds:",subdf['AvgW'].mean())
    for i in range(subdf.shape[0]):
        gain = gain+10*(subdf['AvgW'][i]-1)

    subdf = df.loc[((df['LRank']<=thr) & (df['WRank']>df['LRank']+delta) & (df['Series']==series) & (df['Round']==r))]
    print("loss:",subdf.shape[0]," average odds:",subdf['AvgW'].mean())
    nmatch = nmatch+subdf.shape[0]
    gain = gain-10*subdf.shape[0]
    
    gains[k] = gain
    gainspermatch[k] = gain/nmatch
    roi[k] = gain/(nmatch*10)*100
    print("investment:",nmatch*10," gain:",gain," ROI:",roi[k])
    k=k+1
plt.rcParams['figure.figsize'] = [10, 6]
# show total gains per round in grand slams for top 10 players against theoretically weakest players
plt.bar(round,gains)
plt.ylabel("Gains for 10$ bet on each game (GS, top 10 vs higher rank)")
plt.show()<jupyter_output><empty_output><jupyter_text>The data indicates that highest gains are made when betting with the odds in 2nd round and Semifinals. In 2nd round, top 10 players are generally opposed to much higher ranked players (ranked 50-100), explaining why the odds are lower than in semifinals where top 10 players are expected and therefore playing against each others, reason why the odds are higher and gains as well (despite the fact there are only 2 semifinals per tournament, compared to normally 10 matches in 2nd round for top 10 players)<jupyter_code>plt.bar(round,gainspermatch)
# show gain per match per round in grand slams for top 10 players against theoretically weakest players
plt.ylabel("Gains per match for 10$ bet on each game (GS, top 10 vs higher rank)")
plt.show()<jupyter_output><empty_output><jupyter_text>For the reasons explained above, this data shows clearly confirms that the gain per match is much higher for semifinals than 2nd round.<jupyter_code>plt.bar(round,roi)
# show return on investment per round in grand slams for top 10 players against theoretically weakest players
plt.ylabel("Return on investment in % (GS, top 10 vs higher rank)")
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13332601777680236, 'fraction_numerical': 0.04685613958081861, 'mean_word_length': 5.307266435986159, 'pattern_counts': {'":': 1, '<': 35, '<?xml version=': 0, '>': 41, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 5, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '259647', 'license': 'MIT', 'n_tokens_mistral': 3452, 'n_tokens_neox': 3106, 'n_words': 958, 'path': 'Analysis.ipynb', 'repo_name': 'leosartorius/udacity', 'size': '9113'}