<jupyter_start><jupyter_text>Linear Regression: Moneyball A statistical case study of the popular sports storyOverviewOne of the most compelling stories of sports analytics made popular by the 2011 movie Moneyball, the Oakland Athletics team of 2002 created history by winning 20 consecutive games between August 13 and September 2002.Much of the Oakland Athletics (popularly referred to as the A’s) success in that season is attributed to their General Manager, <NAME> and former Assistant General Manager, <NAME>.DePodesta, a graduate in Economics from Harvard University joined the A’s in 1999 and quickly started incorporating analysis of baseball statistics to value and purchase players.(Sabermetrics)This approach came into the limelight following the historical 2002 season when <NAME> purchased vastly undervalued players to supplant the departure of key players prior to the start of the season. Due to severe budgetary constraints of the Oakland Athletics, Billy resorted to the methods being researched by DePodesta in order to purchase underrated players to meet the California based team’s payroll.s This code attempts to recreate portions of Paul DePodesta’s analysis in order to demonstrate the power of the simplest supervised learning technique a.k.a Linear Regression.This analysis is very similar to the case study in MIT’s Analytics EdgeLinear regression equation:![image.png](attachment:image.png)Dataset: https://www.kaggle.com/wduckett/moneyball-mlb-stats-19622012/data<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
df = pd.read_csv("/Users/swaruptripathy/Desktop/Data Science/baseball.csv")
df.head()<jupyter_output><empty_output><jupyter_text>Here:RA stands for runs allowed.RS stand for runs scored.OBP stands for On Base Percentage.SLG stand for Slugging Percentage.BA is Batting Average.OOBP is opponent’s OBP.OSLG is opponent’s SLG.W stands for number of wins in that season.A new variable RD is added as the difference between RS and RA.Additionally, we shall only keep records with Year < 2002 in order to replicate the data prior to the 2002 season.We will use plots in order to find insights regarding the distributions of the numerical variables.<jupyter_code>len(df.columns)
len(df)
df.loc[:,'RD'] = df.RS - df.RA
df.head()
moneyball = df.loc[df['Year']<2002]
moneyball.head()
f, ax = plt.subplots(2,2)
ax[0,0].hist(moneyball.RS,bins = 50)
ax[0,0].set_title("Runs Scored")
ax[0,1].hist(moneyball.RA,bins = 50)
ax[0,1].set_title("Runs Allowed")
ax[1,0].hist(moneyball.OBP,bins = 50)
ax[1,0].set_title("On Base Percentage")
ax[1,1].hist(moneyball.SLG,bins = 50)
ax[1,1].set_title("Slugging Percentage")
f.subplots_adjust(hspace=0.2)
f.set_figheight(10)
f.set_figwidth(10)<jupyter_output><empty_output><jupyter_text>Observe that all four features(variables) are approximately normally distributed. Both, the runs scored and runs allowed graph seems to be skewed slightly towards the left. The tails of the runs scored graph seems to be thicker than that of the runs allowed graph.https://www.youtube.com/watch?v=KWPhV6PUr9o&t=85sAccording to DePodesta’s character in the clip, the A’s required at least 99 wins in order to make the playoffs in 2002. Lets try to visualise this insight.<jupyter_code># Subsetting teams qualified for playoffs
moneyball1 = moneyball.loc[moneyball['Playoffs']==1]
moneyball0 = moneyball.loc[moneyball['Playoffs']==0]
# Plotting scatterplot
plt.figure(figsize=(10,8))
plt.scatter(x=moneyball0.W,y = moneyball0.RS,c="red")
plt.scatter(x=moneyball1.W,y = moneyball1.RS,c="blue")
plt.title("Wins vs Runs Scored")
plt.xlabel("Wins")
plt.ylabel("Runs Scored")
plt.axvline(x = 99)<jupyter_output><empty_output><jupyter_text>From the above plot we see that DePodesta’s estimate of 99 wins makes sense since there appears to be only 3 observations (red) who haven’t made it to the playoffs with ≥ 99 wins.DePodesta also calculated that the A’s need to score atleast 814 runs and allow only 645 runs in order to make it to the postseason. That meant that the difference between runs scored and runs allowed needed to be approximately 169(RD). Let us look at the relationship between RD and Wins.<jupyter_code>x = np.array(moneyball.RD)
y = np.array(moneyball.W)
# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]
#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("Run Difference")
plt.ylabel("Wins")
plt.show()
# We can clearly see that the relationship between the two variables is linear.
print(np.corrcoef(x,y))<jupyter_output>[[1.         0.93851498]
 [0.93851498 1.        ]]<jupyter_text>Additionally, there is a strong and positive correlation of 0.9385 between RD and number of wins.Thus, our exploratory data analysis confirms that DePodesta’s estimates have been spot on so far. Now in the study we shall build regression models to further validate DePodesta’s analysis. we will study relationships between the numerical variables and build a few models to predict the Oakland A’s run in 2002.Now let us look at the formulas for calculating some of the numerical terms that may seem obscure to most people. On Base Percentage:![image.png](attachment:image.png)Formula for calculating On-base Percentage. SourceH : HitsB: Base on balls (Walks)HBP: Hits by pitchAB: At batSF: Sacrifice fly Slugging Percentage:![image.png](attachment:image.png)Formula for calculating Slugging percentage. Source1B, 2B, 3B: singles, doubles, triplesHR: Home RunAB: At bat Batting Average:![image.png](attachment:image.png)Formula for calculating Batting average. SourceH: HitsAB: At batWe can see that all three of the statistics previously discussed contribute in some way to the number of runs scored by the team. We won’t go into specific details of explaining each variable involved. All we need to know is that OBP, SLG and AVG are strong indicators of runs scored by the particular team.Let us validate these assumptions by plotting the relationship between each of these variables with RS(Runs scored):<jupyter_code>x = np.array(moneyball.OBP)
y = np.array(moneyball.RS)

# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]

#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("On-base percentage")
plt.ylabel("Runs scored")
plt.show()
print(np.corrcoef(moneyball.OBP,moneyball.RS))
x = np.array(moneyball.SLG)
y = np.array(moneyball.RS)

# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]

#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("Slugging percentage")
plt.ylabel("Runs scored")
plt.show()
print(np.corrcoef(moneyball.SLG,moneyball.RS))
x = np.array(moneyball.BA)
y = np.array(moneyball.RS)

# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]

#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("Batting average")
plt.ylabel("Runs scored")
plt.show()
print(np.corrcoef(moneyball.BA,moneyball.RS))<jupyter_output>[[1.         0.83162475]
 [0.83162475 1.        ]]<jupyter_text>There is a clear linear relationship between each of the variables and runs scored.As a rule of thumb, correlation above 0.7 is considered as strong positive correlation. All of our variables satisfy this rule.Similarly, we can hypothesise that OOBP(Opponent’s OBP) and OSLG(Opponent’s SLG) have a similar relationship with RA(Runs allowed).Before we see the relationship between these variables and RA, I noticed that there are a few missing values in the OOBP and OSLG columns. We have to remove these rows of observations in order to plot a proper relationship.<jupyter_code>moneyball.isnull().sum()
moneyball.drop(['RankSeason','RankPlayoffs'],axis = 1,inplace = True)
moneyballnew = moneyball.dropna()
moneyballnew.head(5)
x = np.array(moneyballnew.OOBP)
y = np.array(moneyballnew.RA)

# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]

#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("Opponent's On-base percentage")
plt.ylabel("Runs allowed")
plt.show()
print(np.corrcoef(moneyballnew.OOBP,moneyballnew.RA))
x = np.array(moneyballnew.OSLG)
y = np.array(moneyballnew.RA)

# Deriving slope,intercept values
slope, intercept = np.polyfit(x, y, 1)
abline_values = [slope * i + intercept for i in x]

#Plotting the figure
plt.figure(figsize=(10,8))
plt.scatter(x, y)
plt.plot(x, abline_values, 'b')
plt.title("Slope = %s" % (slope))
plt.xlabel("Opponent's Slugging percentage")
plt.ylabel("Runs allowed")
plt.show()
print(np.corrcoef(moneyballnew.OOBP,moneyballnew.RA))<jupyter_output>[[1.         0.90990783]
 [0.90990783 1.        ]]<jupyter_text>Model BuildingWe shall now build regression models in order to predict runs scored, runs allowed and wins.The scikit-learn library in python is extremely powerful in building a vast variety of machine learning models using very few lines of code. We shall utilise this library in order to build our models.Our first model is used to predict ‘runs scored’. Our independent variables as on-base percentage, slugging percentage, batting average.<jupyter_code>from sklearn.linear_model import LinearRegression
from sklearn.cross_validation import train_test_split
# Extracting our variables from the dataframe.
x = moneyball[['OBP','SLG','BA']].values
y = moneyball[['RS']].values

model = LinearRegression()
model.fit(x,y)
print(model.intercept_)
print(model.coef_)<jupyter_output>[-788.45704708]
[[2917.42140821 1637.92766577 -368.96606009]]<jupyter_text>Thus, our model is of the following form:RS = -788.46+2917.42×(OBP)+1637.93×(SLG)-368.97×(BA)At this point, we notice something odd. Our model has a variable with a negative coefficient a.k.a Batting Average. This is counter-intuitive as a team with a higher batting average should have more runs scored. This situation represents a phenomenon known as multi-collinearity which leads to discrepancies in multiple linear regression models.In order to avoid multi-collinearity, we must either add an interaction variable or remove the variable causing the discrepancy. For the sake of simplicity let us remove BA from our model.<jupyter_code>x = moneyball[['OBP','SLG']].values
y = moneyball[['RS']].values

# Calling our model object.
RS_model = LinearRegression()

# Fitting the model.
RS_model.fit(x,y)

# Printing model intercept and coefficients.
print(RS_model.intercept_)
print(RS_model.coef_)

RS_model.predict([[0.339,0.430]])<jupyter_output>[-804.62706106]
[[2737.76802227 1584.90860546]]<jupyter_text>Our refined model is:RS = -804.63+2737.77×(OBP)+1584.91×(SLG)Similarly, we build a model for ‘runs allowed’ using opponent’s on-base percentage and opponent’s slugging percentage as independent variables:<jupyter_code># Extracting our variables from the dataframe.
x = moneyballnew[['OOBP','OSLG']].values
y = moneyballnew[['RA']].values
# Calling our model object.
RA_model = LinearRegression()
# Fitting the model.
RA_model.fit(x,y)
# Printing model intercept and coefficients.
print(RA_model.intercept_)
print(RA_model.coef_)<jupyter_output>[-837.37788861]
[[2913.59948582 1514.28595842]]<jupyter_text>Model for runs allowed:RA = -837.38 + 2913.60 ×(OOBP) + 1514.29 ×(OSLG)Lastly, our model for predicting wins from runs difference:<jupyter_code># Extracting our variables from the dataframe.
x = moneyball[['RD']].values
y = moneyball[['W']].values
# Calling our model object.
W_model = LinearRegression()
# Fitting the model.
W_model.fit(x,y)
# Printing model intercept and coefficients.
print(W_model.intercept_)
print(W_model.coef_)<jupyter_output>[80.88137472]
[[0.10576562]]<jupyter_text>Model for wins:W = 80.88 + 0.11 ×(RD) Model PredictionsHere are the Oakland Athletics statistics in 2001 before the playoffs.OBP: 0.339SLG: 0.430OOBP: 0.307OSLG: 0.373Lets plug in these values into the above models to generate predictions.<jupyter_code># Prediction for runs scored.
RS_model.predict([[0.339,0.430]])
# Predictions for runs allowed.
RA_model.predict([[0.307,0.373]])<jupyter_output><empty_output><jupyter_text>Thus, our models predicted the following:RS ≈ 805RA ≈ 622This implies that RD = 183.Finally, we plug in RD into our wins model:<jupyter_code># Prediction for wins.
W_model.predict([[183]])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10272161741835148, 'fraction_numerical': 0.03631415241057543, 'mean_word_length': 5.7159268929503915, 'pattern_counts': {'":': 0, '<': 47, '<?xml version=': 0, '>': 45, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '641285', 'license': 'MIT', 'n_tokens_mistral': 4581, 'n_tokens_neox': 4056, 'n_words': 1415, 'path': 'Moneyball.ipynb', 'repo_name': 'swaruptripathy/DataScience-Python', 'size': '12878'}