<jupyter_start><jupyter_text>**Prediction of Football matches**  **Overview**You have been recruited as a football analyst in a company - Mchezopesa Ltd and tasked to accomplish the task below.A prediction result of a game between team 1 and team 2, based on who's home and who's away, and on whether or not the game is friendly (include rank in your training).You have two possible approaches (as  shown below) given the datasets that will be providedInput: Home team, Away team, Tournament type (World cup, Friendly, Other)  **Defining the question**1.Predict the number of goals scored by the home team.2.Predict the number of goals scored by the away team.3.Predict the outcome of a football match between team 1 and team 2.  **Metric of Succcess**1.A polynomial regression model that predicts the number of goals scored by the home team with above 50% accuracy.2.A polynomial regression model that predicts the number of goals scored by the away team with an accuracy of above 50%.3.A logistic regression model that predicts the outcome of a football match between team 1 and team 2.4.Quantile, lasso and ridge regression models that predict the outcome of a football match. **Context**We are given the dataset to help us understand and predict number of goals scored by the home team and the number of goals scored by the away team.  **Experimental Design**I will use the CRISP-DM methodology to tackle this question.Here is the flow:1.Problem Understanding2.Data Understanding3.Data Preparation4.Modelling5.Evaluation  ***Importing the Libraries***<jupyter_code>#Ill first import the necessary libraries for this dataset
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import datetime
import statsmodels.api as sm
import statsmodels.formula.api as smf
from sklearn.preprocessing import PolynomialFeatures
from sklearn.linear_model import LinearRegression, LogisticRegression, Ridge, Lasso
from sklearn.metrics import r2_score, mean_squared_error, accuracy_score, confusion_matrix
from sklearn.model_selection import train_test_split, KFold, cross_val_score, GridSearchCV
import warnings
warnings.filterwarnings('ignore')
#ILl then load the dataset
df_1 = pd.read_csv("/content/fifa_ranking.csv")
df_1.head()
#Ill then preview the second dataset,the first 4
df_2 = pd.read_csv("/content/results.csv")
df_2.head()
#Ill then create a copy of the 2 datasets
df1_copy = df_1.copy
df2_copy = df_2.copy<jupyter_output><empty_output><jupyter_text>***Data Cleaning***<jupyter_code>#Ill check if their is null values in the dataset
df_1.isnull().sum()
#Their are no null values in the dataset
#Ill then check for nulls in the second dataset 
df_2.isnull().sum()
#Their s no null values in this dataset
#Ill then check the column names in first dataset
df_1.columns
#Ill then check the column names in second dataset
df_2.columns

#Ill then check the iformation of the dataset
df_1.info()
#Ill then check the information of the other dataset
df_2.info()
#ILl then check the datatypes of the first dataset(fifa dataset)
df_1.dtypes
#For appropriate reasons
#Ill then check the datatype of the results dataset
df_2.dtypes
#Ill then filter out the dataset emmiting neutral grounds
df_2 = df_2[df_2.neutral == False]
df_2.head()



#Ill then work with the date column to convert it to the day, the month and the year
df_2['year'] = pd.DatetimeIndex(df_2['date']).year
df_2['month'] = pd.DatetimeIndex(df_2['date']).month
df_2['day'] = pd.DatetimeIndex(df_2['date']).day


#ILl then confirm if it has been converted
df_2.head(2)
#It has been converted
#Ill then drop the date column since we don't need it anymore
#Ill also drop the neutral column since its "False" in every game
df_2 = df_2.drop(['date','neutral'], 1)
df_2
#They have been dropped
#Ill then create an algorithim that will determine the outcome of the football match

outcome = []

for i in range(len(df_2['home_team'])):
  if df_2.home_score.iloc[i] > df_2.away_score.iloc[i]:
    outcome.append('Win')
  elif df_2['home_score'].iloc[i] < df_2.away_score.iloc[i]:
    outcome.append('Lose')
  else:
    outcome.append('Draw')
  
#ILl then create a new column for the outcome
df_2['outcome'] = outcome
df_2.apply(lambda col: col.drop_duplicates().reset_index(drop=True))
#ILl then preview if the algorithim has worked
df_2.head(10)
#It has worked

#Ill then split the rank_date column into day, month and year in the fifa_ranking dataset
df_1['year'] = pd.DatetimeIndex(df_1['rank_date']).year
df_1['month'] = pd.DatetimeIndex(df_1['rank_date']).month
df_1['day'] = pd.DatetimeIndex(df_1['rank_date']).day

#ILl then confirm if the date has been split.
df_1.head(6)
#Ill then drop unnecessary columns in fifa_ranking dataset
df_1.drop(['country_abrv','rank_date', 'total_points', 'cur_year_avg', 'cur_year_avg_weighted' ,'last_year_avg' , 'last_year_avg_weighted' , 'two_year_ago_avg', 'two_year_ago_weighted', 'three_year_ago_avg' ,'three_year_ago_weighted', 'confederation'], axis =1, inplace = True)
#Ill then preview the dataset to see if they have been dropped
df_1.head()
#Ill then drop the rank change in the dataset
df_1.drop(['rank_change'],axis=1)<jupyter_output><empty_output><jupyter_text>**Filtering out the dataset**<jupyter_code>df_2.head()
#Checking for unique values in the tournament column
df_2.tournament.unique()
#Ill work with relevant tournament fields
#Ill then create a function that will hold an algorithim for selected tournament type
def excluding(tournament):
  if tournament == 'Friendly':
    return 'Friendly'
  elif tournament == 'FIFA World Cup':
    return 'World Cup'
  else:
    return 'Other'

df_2['tournament'] = df_2.tournament.apply(lambda x: excluding(x))
df_2.head()
#Ill then confirm the unique tournaments
filtered_results_df = df_2.tournament.unique()
filtered_results_df
#Ill also filter out the years in results dataset
filtered_results_df = df_2[df_2['year'].between(2006,2018, inclusive=True)]
filtered_results_df.head()

filtered_results_df.shape
#8776 rows and 11 columns
#Ill then filter  out the first dataset in years as well
filtered_ranking_df = df_1[df_1['year'].between(2006,2018, inclusive=True)]
filtered_ranking_df.head()

filtered_ranking_df.shape<jupyter_output><empty_output><jupyter_text>***Merging the datasets***<jupyter_code>#Ill then merge the 2 datasets for analysis and prediction
merged = pd.merge(filtered_ranking_df, filtered_results_df, how='left', left_on=['month','year'], right_on=['month', 'year'])

#ILl then preview the merged dataset
merged.head(10)
merged.shape<jupyter_output><empty_output><jupyter_text>**Cleaning the Merged dataset**<jupyter_code>#Ill then drop some irrelevant column names
merged = merged.drop(['rank_change'],axis = 1)
#Ill also drop other irrelevant column names
merged = merged.drop(['day_x','day_y'],axis=1)
#Ill then check the column names in the dataset
merged.columns
#Ill then check the shape of the dataset
merged.shape
#The merged dataset has 3157060 rows × 16 columns
#Ill then check the column names of the dataset
merged.columns

#Ill then check for duplicated values
merged.duplicated().sum()
#The dataset has 3 thousand duplicated values
#Ill then proceed to drop the duplicated values
merged.drop_duplicates(inplace=True)
merged.duplicated().sum()
#Ill then confirm if they have been droppped

#ILl then get the ranks of the home team and away team

home_rank = merged[merged.home_team == merged.country_full]
home_rank.head()
#Here I have selected observations equal to country so as to get the home ranks

#Renaming the columns
home_rank = home_rank.rename(columns={'rank':'home_rank'})
home_rank.head()
#Ill then do the same for the away team
away_rank = merged[merged.away_team == merged.country_full]
away_rank.head()
away_rank = away_rank.rename({'rank': 'away_rank'}, axis=1)
away_rank.head()
away_rank.drop('country',1,inplace=True)
#Ill then merge the new home ranks and away ranks
tired_of_Nans = pd.merge(home_rank,away_rank,how="left", on = ['away_team', 'year', 'month'])
tired_of_Nans<jupyter_output><empty_output><jupyter_text>**Cleaning the Newly merged dataset**<jupyter_code>tired_of_Nans.isnull().sum()
tired_of_Nans.dropna(inplace=True)
tired_of_Nans.duplicated().sum()
tired_of_Nans.head()

#Ill then drop unnecesary columns in the dataset
tired_of_Nans.drop(['country_full_x','previous_points_x',
        'previous_points_y','home_team_y',	'home_score_y',	'away_score_y',	'tournament_y',	'city_y',
       		'outcome_y'], axis = 1, inplace = True)
tired_of_Nans.columns
tired_of_Nans.shape
#Ill then drop other irrelevant columns in the dataset
tired_of_Nans.drop(['city_x','country_full_y'],axis=1,inplace=True)
tired_of_Nans.head()
#Ill then rename some of the columns in the dataset
tired_of_Nans = tired_of_Nans.rename({'home_team_x': 'home_team','home_score_x':'home_score',
                                      'away_score_x':'away_score','tournament_x':'tournament',
                                      'outcome_x':'outcome'},axis=1)

#Ill then confirm if it has been renamed
tired_of_Nans.head()<jupyter_output><empty_output><jupyter_text>**Exploratory Data Analysis**<jupyter_code>#Ill then preview the dataset again
tired_of_Nans.shape
#Ill use a boxplot to check for outliers in home_score and away_score
fig,[ax1,ax2] = plt.subplots(1,2, figsize=(12,5))
sns.boxplot(tired_of_Nans.home_score, ax=ax1)
sns.boxplot(tired_of_Nans.away_score, ax=ax2)

plt.show()
#Their is presence of outliers

#Ill then take out the outliers 
#Ill also plot a boxplot to check if their are still their

Q1 = merged.quantile(0.25)
Q3 = merged.quantile(0.75)
IQR = Q3 - Q1
df_3 = tired_of_Nans[~((tired_of_Nans< (Q1 - 1.5 * IQR)) | (tired_of_Nans > (Q3 + 1.5 * IQR)))]

fig,[ax1,ax2] = plt.subplots(1,2, figsize=(12,5))
sns.boxplot(df_3.home_score, ax=ax1)
sns.boxplot(df_3.away_score, ax=ax2)
fig.suptitle('Boxplots Showing Outliers in Goals Scored', fontsize=15, color='b')

plt.show()
#Their are no presence of outliers
#Ill use a histogram to plot the difference between home goals and away goals.
fig, [ax1,ax2] = plt.subplots(nrows=1,ncols=2, figsize=(10,6))
sns.distplot(df_3['home_score'], bins=10, ax =ax1, kde=False, color='yellow')
sns.distplot(df_3['away_score'], bins=10, ax =ax2, kde=False, color ='green')
fig.suptitle('Distribution of Goals', fontsize=19, color='black')
plt.show()<jupyter_output><empty_output><jupyter_text>***Coorelation***<jupyter_code>#ILl then plot a scatter plot showing Correlation Between Rank and Goals Scored
fig, [axx1, axx2] = plt.subplots(nrows=1, ncols=2, figsize=(12,8))
fig.suptitle('Scatter Showing Correlation Between Rank and Goals Scored', fontsize=19, color='black')
axx1.scatter(df_3.home_rank, df_3.home_score)
axx1.set_xlabel('Home Rank')
axx1.set_ylabel('Home Score')
axx2.scatter(away_rank.away_rank, away_rank.away_score )
axx2.set_xlabel('Away Rank')
axx2.set_ylabel('Away Score')

plt.show()
#Ill then check for nulls in the dataset
df_3.isnull().sum()
#ILl then drop the nulls
df_3.dropna(inplace=True)
#Ill then confirm if it has been dropped
df_3.isnull().sum()<jupyter_output><empty_output><jupyter_text>**Mulitcollinearity**<jupyter_code>#Ill then check for correlation

correlation = df_3.corr()
correlation

#Ill then calculate the VIF score for each of the numerical varaible
pd.DataFrame(np.linalg.inv(correlation.values), index=correlation.index, columns=correlation.columns)<jupyter_output><empty_output><jupyter_text>***Conclusion***Upon detecting Multicolinearity,the VIF scoreshows that the variables is not between 5 and 10.Therefore no correlation.  **Polynomial Regression**<jupyter_code>#Ill create a model that predicts the goals scored by home team
model_Polynomial = df_3[['home_rank','home_score']]
model_Polynomial.head()
#Ill then show the target variables
fitting1 = model_Polynomial.home_rank.values.reshape(-1,1)

greg = np.arange(fitting1.min(), fitting1.max(), 1)[:, np.newaxis]
katono= model_Polynomial.home_score.values.reshape(-1,1)
#Ill then plot a scatter plot to find if their is any relationship
plt.scatter(fitting1, katono, color='blue') 
plt.title('Rank vs Score')
plt.xlabel('Rank')
plt.ylabel('Score')
plt.show()
#No relationship shown
#Ill then split the features to training and test
X_train, X_test, Y_train, Y_test = train_test_split(fitting1,katono, test_size=0.3, random_state=23)

#ILl then create features, polynomial and quadratic
#Ill also add Linear Regression
quadratic = PolynomialFeatures(degree=2)
cubic = PolynomialFeatures(degree=3)
x_quad = quadratic.fit_transform(X_train)
x_test = quadratic.fit_transform(X_test)
x_cube = cubic.fit_transform(X_train)
lin= LinearRegression()
quad = lin.fit(X_train, Y_train)
y_lin = lin.predict(greg)
l_r2 = r2_score(Y_test, lin.predict(X_test))
#Fitting
linpol = lin.fit(x_quad, Y_train)
y_quad_fit = linpol.predict(quadratic.fit_transform(greg))

# calculating R squared
polpre = linpol.predict(x_quad)
quad_r2 = r2_score(Y_train, polpre)
print('Quad RMSE: ', np.sqrt(mean_squared_error(Y_train, polpre)), '\nQuad r_squared: ', quad_r2)
# getting the residuals to check for heteroskedasticity 
resid = np.subtract(polpre, lebo_train)
pd.DataFrame(resid)

resid.mean()
# Ill then get the residuals to check for heteroskedasticity 
residuals= np.subtract(polpre, Y_train)
pd.DataFrame(residuals)

residuals.mean()
#Ill then plot the residuals
plt.scatter(polpre, residuals, color='blue')
plt.ylabel('RESIDUAL')
plt.xlabel('FITTED VALUES')
plt.axhline(y= residuals.mean(), color='red', linewidth=1)
plt.show()<jupyter_output><empty_output><jupyter_text>***Conclusion***On calculating the RMSE, we found that it is 1.2449639664224987. Therefore meaning the model is not good for predicting data since RMSE values between 0.2 and 0.5 shows that the model can relatively predict the data accurately. In addition, Adjusted R-squared more than 0.75 is a very good value for showing the accuracy.IN OUR CASE IT IS A HIGH RMSE  ***Logistice Regression Approach***<jupyter_code>#Creating a copy of the dataframe
very_tired_of_Nulls = tired_of_Nans
#Ill then view the dataframe
very_tired_of_Nulls.head()

logistic = very_tired_of_Nulls.drop(['outcome', 'home_score', 'away_score'], 1)
regression = very_tired_of_Nulls['outcome']
# Ill then create dummy variables to convert categorical columns to numerical variables
dummy = pd.get_dummies(logistic, drop_first=True)
dummy.head()
#Ill then split the datset into train and test
x_train, x_test, y_train, y_test = train_test_split(dummy,regression, test_size=0.2, random_state=123)

#Ill then initialise the model

greg = LogisticRegression()
# training the model on the training set
greg.fit(x_train, y_train)
#Ill start making predictions with the model
predict = greg.predict(x_test)
predict

#Evaluating the perfomance of the model
print('Logistic Model Accuracy: {:.4f}' .format(accuracy_score(y_test, predict)))

print('Confusion Matrix: \n{}' .format(confusion_matrix(y_test, predict)))<jupyter_output>Logistic Model Accuracy: 0.6064
Confusion Matrix: 
[[  0 120 282]
 [  0 288 200]
 [  2  97 792]]<jupyter_text>***The accuracy of the output when using logistic Model shows a 60% accuracy***<jupyter_code>#Ill then evalute the perfomance of the model
train_accuracy = greg.score(x_train, y_train)
test_accuracy = greg.score(x_test, y_test)

print('Training set accuracy: %.4f' % train_accuracy)
print('Test set accuracy: %.4f' % test_accuracy)<jupyter_output>Training set accuracy: 0.5885
Test set accuracy: 0.6064<jupyter_text>***The developed logistic model has 58% accuracy on the training set and 60% accuracy on the test set.***  ***Hyperparemeter Tuning***<jupyter_code>#Cross Validation
Hyper =LogisticRegression(C=1, penalty='l1', solver='liblinear')

#Ill then fit the model
Hyper.fit(x_train,y_train)

#Ill then get the accuracy score of the model
Hyper.score(x_train, y_train)

folds = KFold(n_splits=10, random_state=34)

# getting the cross validation score
from sklearn.model_selection import cross_val_score

selection = cross_val_score(Hyper, dummy, regression, cv=folds, scoring='accuracy')
print(selection.mean())
# instantiating the logistic regression model
Grid_search= LogisticRegression()

# defining the paramenters/hyperparameters
penalty = ['l2']
dual = [True,False]
max_iter = [100,120,130]
C = [1.0,2.0,2.5]

# Creating hyperparameter options
hparameters = dict( penalty=penalty, dual=dual, max_iter=max_iter, C=C)

hparameters<jupyter_output><empty_output><jupyter_text>***Hypertuning Conclusion***Upon doing the tests,the cross validator score was 58%.  ***Quantile Regression***<jupyter_code>tired_of_Nans.head()
sns.pairplot(tired_of_Nans, dropna=True)
#Ill then visualize a pairplot to help in picking the variables suitable for quantile regression


# Ill then use home rank and home_score columns as they are both skewed to the right
c = smf.quantreg('home_score ~ home_rank', tired_of_Nans)
d = c.fit(q=.5)
print(d.summary())<jupyter_output>QuantReg Regression Results                          
==============================================================================
Dep. Variable:             home_score   Pseudo R-squared:           -1.222e-07
Model:                       QuantReg   Bandwidth:                      0.3515
Method:                 Least Squares   Sparsity:                        1.547
Date:                Sat, 17 Jul 2021   No. Observations:                 8904
Time:                        14:39:54   Df Residuals:                     8902
                                        Df Model:                            1
==============================================================================
                 coef    std err          t      P>|t|      [0.025      0.975]
------------------------------------------------------------------------------
Intercept      1.0000      0.015     65.194      0.000       0.970       1.030
home_rank  -3.477e-07      0.000     -0.002      0.9[...]
{'fraction_non_alphanumeric': 0.11459380936417476, 'fraction_numerical': 0.023466309084813945, 'mean_word_length': 5.236585365853658, 'pattern_counts': {'":': 0, '<': 56, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '341885', 'license': 'MIT', 'n_tokens_mistral': 6172, 'n_tokens_neox': 5777, 'n_words': 1792, 'path': 'Moringa_School_Independet_Project_WK_6_Core_2021.ipynb', 'repo_name': 'Katonokatono/Football', 'size': '17898'}