<jupyter_start><jupyter_text>REGRESSION MODEL NOTES We Can Conduct a few different version of this regression model by changing the dependent and independent variables**Dependent Variable**We can elect to make the dependent variable round score or tournament score*Round Score* - would give us more data points, but could also cause higher variation*Tournament Score* - would seem to be the better fit, but we may not have enough data pointsThe dependent variable can also refer to tournament score across all tournaments, or for a specific tournament**Independent Variables**4 major groups of Independent Variables*Greens In Regulation* : Describes how frequently the player makes in to the green at least 2 strokes away from par based on a number of situation. Evaluates a players skill in the fairways/middle game    Consists of ['GIR_PCT_FAIRWAY_BUNKER', 'GIR_PCT_FAIRWAY', 'GIR_PCT_OVERALL', 'GIR_PCT_OVER_100', 'GIR_PCT_OVER_200','GIR_PCT_UNDER_100', 'GREEN_PCT_SCRAMBLE_SAND', 'GREEN_PCT_SCRAMBLE_ROUGH']     *Tee Box*: Describes different elements of a players driving/tee shots. Evaluates a players skill off the tee/long game    Consists of ['TEE_AVG_BALL_SPEED', 'TEE_AVG_DRIVING_DISTANCE', 'TEE_DRIVING_ACCURACY_PCT', 'TEE_AVG_LAUNCH_ANGLE', 'TEE_AVG_LEFT_ROUGH_TENDENCY_PCT', 'TEE_AVG_RIGHT_ROUGH_TENDENCY_PCT', 'TEE_AVG_SPIN_RATE']    *Putting*: Describes a players performance on the green. Evaluates a players putting skill/short game    Consists of ['PUTTING_AVG_ONE_PUTTS', 'PUTTING_AVG_TWO_PUTTS','PUTTING_AVG_PUTTS','PUTTING_AVG_DIST_BIRDIE_INCH']    *Performance Based*: Descibes a players performance in terms of previous results and scores. Evaluates a players consistency and past performances    Consists of ['Par3Average','Par4Average', 'Par5Average', 'HolesPerBirdie', 'HolesPerBogey','FINISHES_TOP10']    **Independence Between Variables**To avoid creating bias in the regression model, we should avoid using the following highly coorelated independent variables together in the same model*GIR*: (GIR_PCT_OVERALL: GIR_PCT_OVER_100, GIR_PCT_FAIRWAY)*Tee*: (TEE_AVG_BALL_SPEED: TEE_AVG_DRIVING_DISTANCE)*Putting*: (PUTTING_AVG_ONE_PUTTS: PUTTING_AVG_TWO_PUTTS : PUTTING_AVG_PUTTS)*Performance Based*: (Par4Average: HolesPerBogey)<jupyter_code># Lets Start with the Dependent Variable as Round Score across all tournaments

roundsDf = pd.read_csv("/dbfs/FileStore/karbide/RoundsReg.txt")

playerStats = pd.read_csv("/dbfs/FileStore/karbide/PlayerStatsComplete.txt")

roundsDf.drop(["Unnamed: 0"], axis = 1, inplace = True)
playerStats.drop(["Unnamed: 0"], axis = 1, inplace = True)
roundScores = roundsDf[["PlayerID","RoundScore"]]
roundsReg = roundScores.merge(playerStats, how = "left", on = "PlayerID")
roundsReg.corr()
# none or the variables are highly coorelated with RoundScore but the performance based ones score the highest
from sklearn.model_selection import train_test_split
from sklearn import linear_model
from sklearn.metrics import r2_score
#selecting Independet Variables (X)
X = roundsReg[["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH","PUTTING_AVG_PUTTS","TEE_AVG_DRIVING_DISTANCE","TEE_DRIVING_ACCURACY_PCT", "FINISHES_TOP10", "GIR_PCT_OVERALL", "GIR_PCT_FAIRWAY_BUNKER"]]
Y = roundsReg[["RoundScore"]]
X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.33)
reg = linear_model.LinearRegression()
reg.fit(pd.DataFrame(X_train),pd.DataFrame(Y_train))
pred = reg.predict(X_test)
err = pd.Series(Y_test["RoundScore"]) - [p[0]for p in pred]
display(err.hist(bins=100))
# seems we get some really crazy predictions
predDf = pd.DataFrame(pred)
predDf.describe()
Y_test.describe()
reg.score(pd.DataFrame(X_train), pd.DataFrame(Y_train))
#This shows the high variance I was worried about, Lets check accuracy
r2_score(Y_test["RoundScore"],pred)
import statistics as stats
def rmse(errors):
  return(pow(stats.mean([pow(e,2) for e in errors]),0.5))
rmse(err)
# seems we are way off, lets change the dependent variable to tournament score
tournamentScore = roundsDf.groupby(["PlayerID","TournamentID"]).agg({"RoundScore":"sum"})
tournamentScore.reset_index(inplace = True)
#since we doing this across all tournaments, we can drop tournament ID
tournamentScore.drop(["TournamentID"],inplace = True, axis = 1)
t_Reg = tournamentScore.merge(playerStats, how = "left", on = "PlayerID")
t_Reg.corr()
# our coorelation are still getting stronger, but still there is little that is very strongly coorelated
X = t_Reg[["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH","PUTTING_AVG_PUTTS","TEE_AVG_DRIVING_DISTANCE","TEE_DRIVING_ACCURACY_PCT", "FINISHES_TOP10", "GIR_PCT_OVERALL", "GIR_PCT_FAIRWAY_BUNKER"]]
Y = t_Reg[["RoundScore"]]

X_train, X_test, Y_train, Y_test = train_test_split(X,Y, test_size = 0.2)

reg = linear_model.LinearRegression()
reg.fit(pd.DataFrame(X_train),pd.DataFrame(Y_train))
pred = reg.predict(X_test)
err = pd.Series(Y_test["RoundScore"]) - [p[0]for p in pred]
display(err.hist(bins=100))
predDf = pd.DataFrame(pred)
print(predDf.describe())
print(Y_test.describe())

print ("R2 Train")
print(reg.score(pd.DataFrame(X_train), pd.DataFrame(Y_train)))
print("R2 Test")
print(r2_score(Y_test["RoundScore"],pred))
print("RMSE")
print(rmse(err))
def linearReg(ind,dep,split):
  X_train, X_test, Y_train, Y_test = train_test_split(ind,dep, test_size = split)
  
  
  reg = linear_model.LinearRegression()
  reg.fit(pd.DataFrame(X_train),pd.DataFrame(Y_train))
  
  pred = reg.predict(X_test)
  err = pd.Series(Y_test["RoundScore"]) - [p[0]for p in pred]
  
  
  print ("R2 Train")
  print(reg.score(pd.DataFrame(X_train), pd.DataFrame(Y_train)))
  print("R2 Test")
  print(r2_score(Y_test["RoundScore"],pred))
  print("RMSE")
  print(rmse(err))
  return(reg.coef_,reg.intercept_)
  
  
# to make this easier lets make a function
X = t_Reg[["Par4Average","HolesPerBirdie","PUTTING_AVG_PUTTS","TEE_AVG_DRIVING_DISTANCE", "FINISHES_TOP10", "GIR_PCT_OVERALL", "Par5Average", "Par3Average"]]
Y = t_Reg[["RoundScore"]]

c2, i2 = linearReg(X,Y,0.2)
#we cant use different tournament in the dependant variable as is but its possible we can use the mean standardized version

tournamentScoreNorm = roundsDf.groupby(["PlayerID","TournamentID"]).agg({"RoundScore":"sum"})
tournamentScoreNorm.reset_index(inplace = True)
#pull the mean scores for each tournament
meanScores = tournamentScoreNorm.groupby("TournamentID").agg({"RoundScore":"mean"})
meanScores.reset_index(inplace=True)
meanScores.columns = ["TournamentID","Mean"]

tournamentScoreNorm = tournamentScoreNorm.merge(meanScores, how="left", on="TournamentID")

tournamentScoreNorm["NormScore"] = tournamentScoreNorm["RoundScore"] - tournamentScoreNorm["Mean"]
tournamentScoreNorm.drop(["TournamentID","Mean","RoundScore"], axis =1, inplace = True)
tournamentScoreNorm.columns = ["PlayerID","RoundScore"]
t_regNorm = tournamentScoreNorm.merge(playerStats, how ="left", on = "PlayerID")
t_regNorm.corr()
X = t_regNorm[["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH", "PUTTING_AVG_ONE_PUTTS", "TEE_AVG_DRIVING_DISTANCE", "TEE_AVG_LEFT_ROUGH_TENDENCY_PCT", "FINISHES_TOP10", "GREEN_PCT_SCRAMBLE_SAND", "Par5Average", "Par3Average"]]
Y = t_regNorm[["RoundScore"]]

c1, i1 = linearReg(X,Y,0.2)
#normalizing improves our R2 value
# again our scores are pretty bad, but maybe they'll get better if we look at just one tournament
tournamentScore2 = roundsDf.groupby(["PlayerID","TournamentID"]).agg({"RoundScore":"sum"})
tournamentScore2.reset_index(inplace = True)

#lets pick the tournament where we have the most players
roundsDf.groupby("TournamentID").agg({"PlayerID":"nunique"}).sort_values("PlayerID", ascending = False).head(2)
# Tournament 429, the players championship
tournamentScore2 =tournamentScore2.loc[tournamentScore2["TournamentID"] == 429]
tournamentScore2.drop(["TournamentID"], axis = 1, inplace = True)
t2_reg = tournamentScore2.merge(playerStats, how = "left", on = "PlayerID")
t2_reg.corr()
#now we have some much stronger coorelations, lets try to use them
X = t2_reg[["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH", "PUTTING_AVG_ONE_PUTTS", "TEE_AVG_DRIVING_DISTANCE", "TEE_AVG_LEFT_ROUGH_TENDENCY_PCT", "FINISHES_TOP10", "GREEN_PCT_SCRAMBLE_SAND", "Par5Average", "Par3Average"]]
Y = t2_reg[["RoundScore"]]

c3,i3 = linearReg(X,Y,0.2)

#now our R2 is much higher, but can we still do better?
#mean standardize for one tournament
mean429 = meanScores.loc[meanScores["TournamentID"]==429]
mean429 = mean429["Mean"][22]
mean429
Y = t2_reg[["RoundScore"]]
Y["RoundScoreNorm"] = Y["RoundScore"] - mean429
Y = Y[["RoundScoreNorm"]]
Y.columns = ["RoundScore"]
c4,i4 = linearReg(X,Y,0.2)
#our result is about the same
# lets just play w the ivs for a bit
X = t2_reg[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GREEN_PCT_SCRAMBLE_SAND"]]
Y = t2_reg[["RoundScore"]]

c5,i5 = linearReg(X,Y,0.2)
# lets just play w the ivs for a bit
X = t2_reg[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GIR_PCT_OVERALL"]]
Y = t2_reg[["RoundScore"]]

c5,i5 = linearReg(X,Y,0.2)<jupyter_output><empty_output><jupyter_text>Since our most accurate model is specfic tournament results, lets add the 3 Strokes Gained categories<jupyter_code>StrokesGained = pd.read_csv("/dbfs/FileStore/karbide/StrokesGainedIDs.txt")
StrokesGained.drop(["Unnamed: 0"], axis = 1, inplace = True)
t2_regSG = t2_reg.merge(StrokesGained, how = "inner", on = "PlayerID")
t2_regSG.columns
# now with SG
X = t2_regSG[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GREEN_PCT_SCRAMBLE_SAND", 'TOTAL SG:T', "TOTAL SG:T2G", 'TOTAL SG:P']]
Y = t2_regSG[["RoundScore"]]

c6,i6 = linearReg(X,Y,0.2)
X = t2_regSG[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GREEN_PCT_SCRAMBLE_SAND", 'TOTAL SG:T', "TOTAL SG:T2G", 'TOTAL SG:P']]
Y = t2_regSG[["RoundScore"]]

c7,i7 = linearReg(X,Y,0.2)
X = t2_regSG[["GREEN_PCT_SCRAMBLE_SAND", 'TOTAL SG:T', "Par4Average","HolesPerBirdie"]]
Y = t2_regSG[["RoundScore"]]

c8,i8 = linearReg(X,Y,0.2)
print(c8,i8)<jupyter_output><empty_output><jupyter_text>MODEL RESULTS AND COEFFICIENTS**Tournament 429***Model 1*:IVs -> ["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH", "PUTTING_AVG_ONE_PUTTS", "TEE_AVG_DRIVING_DISTANCE", "TEE_AVG_LEFT_ROUGH_TENDENCY_PCT", "FINISHES_TOP10", "GREEN_PCT_SCRAMBLE_SAND", "Par5Average", "Par3Average"]DV -> Tournament ScoreR2 and RMSE = 0.3229964494135511, 4.795109852419135coef, int = [[ 2.32846057e+01 -2.26333978e-01  6.19652183e-02  1.83444680e-01 -2.53906041e-02 -2.46384591e-01 -1.97731802e-02 -1.40051312e-01 1.61535095e+01  1.39734513e+01]], 16.67094541*Model 2*IVs -> ["Par4Average","HolesPerBirdie","PUTTING_AVG_DIST_BIRDIE_INCH", "PUTTING_AVG_ONE_PUTTS", "TEE_AVG_DRIVING_DISTANCE", "TEE_AVG_LEFT_ROUGH_TENDENCY_PCT", "FINISHES_TOP10", "GREEN_PCT_SCRAMBLE_SAND", "Par5Average", "Par3Average"]DV -> Tournament Score/Mean StandardizedR2 and RMSE = 0.27481894608420776, 5.278633537736203coef, int =[[ 6.31478232e+00 -8.16209567e-02  7.18796229e-02 -4.30983221e-01 -8.41258947e-03 -2.12066433e-01  7.78575502e-02 -1.20083842e-01 2.55502279e+01  7.95291971e+00]], 15.26182535*Model 3*IVs -> ["Par4Average","HolesPerBirdie", "HolesPerBogey", "GREEN_PCT_SCRAMBLE_SAND", "Par5Average", "Par3Average"]DV -> Tournament ScoreR2 and RMSE = 0.3787100076428008, 4.605715520235553coef, int = [[ 6.81122888  1.61102777 20.98822676 19.81415731  0.39739065 -0.09967628]], 2.13676949*Model 4*IVs -> [["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GREEN_PCT_SCRAMBLE_SAND", 'TOTAL SG:T', "TOTAL SG:T2G", 'TOTAL SG:P']]DV -> Tournament ScoreR2 and RMSE = 0.5044001176837463,3.52737074470192coef, int = [[-1.46756830e+01  7.27174253e-01  4.24465311e+00 -2.42897881e+00 6.32162133e-02 -1.04608027e-01 -6.62482431e+02  6.62435768e+02 6.62423818e+02]] [4.27898458]<jupyter_code>print(c6)
print(i6)
#Lets try some feature importance to increase our model accuracy
t2_regSG.corr(method = 'pearson')
t2_regSG2 = t2_regSG.drop(["PlayerID"], axis = 1)
t2_regSG2.corr(method = 'pearson')
from sklearn.preprocessing import MinMaxScaler
X = t2_regSG[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey","GREEN_PCT_SCRAMBLE_SAND", 'TOTAL SG:T', "TOTAL SG:T2G", 'TOTAL SG:P','AVERAGE']]
Y = t2_regSG[["RoundScore"]]

scaler = MinMaxScaler()
scaler.fit(X)
sX = scaler.transform(X)
cols = X.columns
X_new = pd.DataFrame(sX,columns = cols)

c7,i7 = linearReg(X,Y,0.2)
print(c7,i7)
#coefficients sugguest we remove some of the variables
X = t2_regSG[["Par4Average","HolesPerBirdie", "Par5Average", "Par3Average", "HolesPerBogey",'AVERAGE']]
Y = t2_regSG[["RoundScore"]]

scaler = MinMaxScaler()
scaler.fit(X)
sX = scaler.transform(X)
cols = X.columns
X_new = pd.DataFrame(sX,columns = cols)

c8,i8 = linearReg(X,Y,0.2)
print(c8,i8)
X = t2_regSG[["Par4Average", "Par5Average", "Par3Average",'AVERAGE']]
Y = t2_regSG[["RoundScore"]]

scaler = MinMaxScaler()
scaler.fit(X)
sX = scaler.transform(X)
cols = X.columns
X_new = pd.DataFrame(sX,columns = cols)

c8,i8 = linearReg(X,Y,0.2)
print(c8,i8)
X = t2_regSG[['TOTAL SG:T', "TOTAL SG:T2G", 'TOTAL SG:P']]
Y = t2_regSG[["RoundScore"]]

scaler = MinMaxScaler()
scaler.fit(X)
sX = scaler.transform(X)
cols = X.columns
X_new = pd.DataFrame(sX,columns = cols)

c9,i9 = linearReg(X,Y,0.2)
print(c9,i9)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1379592443849472, 'fraction_numerical': 0.060092220734791015, 'mean_word_length': 7.653153153153153, 'pattern_counts': {'":': 5, '<': 13, '<?xml version=': 0, '>': 21, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '200126', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 5862, 'n_tokens_neox': 5257, 'n_words': 1042, 'path': 'codeSheets/SEAS6401/PGAProject/Regression_Analysis.ipynb', 'repo_name': 'kylearbide/kylearbide.github.io', 'size': '13446'}