<jupyter_start><jupyter_text>Defining the question  a) Specifying the questionYou have been recruited as a football analyst in a company - Mchezopesa Ltd and tasked to accomplish the task below.A prediction result of a game between team 1 and team 2, based on who's home and who's away, and on whether or not the game is friendly (include rank in your training). You have two possible approaches (polynomial and logistic approaches) given the datasets that will be provided  b) Defining the metrics for successThe objective for this data includes investigating whether or ot the game is friendly while making the predictions  c) Understanding the contextYou have been recruited as a football analyst in a company - Mchezopesa Ltd and tasked to accomplish the task below. ---- Predict the result of a game between team 1 and team 2, based on who's home and who's away, and on whether or not the game is friendly (include rank in your training).  d) Recording the Experimental Design1.  reading in the datasets2. checking the data for duplicates and missinng data 3. dropping of duplicates and removal filling in of missing data values with zero4. Finding and dealing with outliers, anomalies, and missing data within the dataset.5. Performing of EDA recording of observations.6. Implementing the solution by performing the respective analysis i.e. factor analysis, principal component analysis, and discriminant analysis.7. Perform any necessary feature engineering8. Check of multicollinearity9. Start building the model, Cross-validate the model10. Compute RMSE11. Create residual plots for your models, and assess their heteroscedasticity using Bartlett’s test12. Perform appropriate regressions on the data including your justification13. Challenge your solution by providing insights on how you can make improvements    Checking the data<jupyter_code># Importing the libraries
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
# Loading the fifa dataset
fifa = pd.read_csv('/content/fifa_ranking.csv',parse_dates=[-1])
# Previewing the data set
fifa.head(1)
# Looking for the shape of fifa data set
fifa.shape
# Checking for columns of data sets
fifa.columns
# Dropping unnecessary columns
fifa.drop(columns=['cur_year_avg_weighted', 'last_year_avg', 'last_year_avg_weighted',
       'two_year_ago_avg', 'two_year_ago_weighted', 'three_year_ago_avg',
       'three_year_ago_weighted'],inplace= True)
# Dropping more columns
fifa.drop(columns=["country_abrv","total_points","previous_points","rank_change","cur_year_avg"],inplace=True)
#previewing the data set after dropping  unnecessary columns
fifa.head(1)
# checking for info of the dataset
fifa.info()
# creating year column in the data set
fifa['year'] = fifa['rank_date'].dt.year
# Preview the data set
fifa.head(1)
# Loading the results dataset
results = pd.read_csv('/content/results.csv',parse_dates=[0])
results.head(1)
# Checking the shape of results datasets
results.shape
# checking the info of the results dataset
results.info()
# Creating the year columns
results['year'] = results['date'].dt.year
#  checking the unique years
results['year'].unique()
# checking unique years in the dataset
fifa['year'].unique()
# Filtering out the data for 4 years because uses 4 years data for ranking
results = results[results['year'] >= 2014]
results= results[results['year'] < 2019]
# checking number of years in the dataset
results['year'].unique()
# Filtering out the data for 4 years because uses 4 years data for ranking
fifa = fifa[fifa['year'] >= 2014]
# checking number of years in the dataset
fifa['year'].unique()
# merging the two datasets on year
final = results.merge(fifa,left_on=['home_team','year'],right_on=['country_full','year'],how= "inner")
final.head(1)
# merging the two data sets
final = final.merge(fifa,left_on=['away_team','year'],right_on=['country_full','year'],how= "inner")
final.head(1)
# dropping unnecessary columns
final = final.drop(columns=['city','country',"neutral",'year','country_full_x','country_full_y'])
# previewing the datasets
final.head(1)
# dropping rank date x and rank date y
final = final.drop("rank_date_x",axis =1)
final = final.drop("rank_date_y",axis =1)
# checking for duplicates in the datasets
final.duplicated(subset=['date','home_team','away_team','home_score','away_score','tournament']).sum()
# dropping the datasets
final.drop_duplicates(subset=['date','home_team','away_team','home_score','away_score','tournament'], keep= 'first', inplace= True)
# checking for missing values in the data
final.isnull().sum()
# checking for tournaments details
final['tournament'].unique()
# filtering tournament column and creating new column 
def y(match) :
  if match == 'FIFA World Cup':
    return 1
  elif match == 'FIFA World Cup qualification':
    return 2
  else:
    return 3

final['competition']= final['tournament'].apply(lambda x:y(x))
final['competition'].unique()
# Checking info of the dataset
final.info()
 # Renaming the rank x and rank y columns
 final.rename(columns={'rank_x':'home_team_rank', 'rank_y':'away_team_rank'},inplace=True)
# dropping unnecessary columns
final.drop(columns=["confederation_y","confederation_x"],inplace=True)
# previewing the dataset
final.head(1)
# creating new column
def outcome(cols):
  home_score = cols[0]
  away_score = cols[1]
  if home_score > away_score:
        return 'Win'
  elif home_score < away_score:
        return 'Loss'
  else:
        return 'Draw'
      
final['status'] = final[['home_score','away_score']].apply(outcome,axis=1)
# previewing the dataset
final.head(1)<jupyter_output><empty_output><jupyter_text>EXPLORATORY DATA ANALYSIS<jupyter_code># Frequency table for the home team
final['home_team'].value_counts()
# Frequency table for the away team
final["away_team"].value_counts()
# Frequency table for the tournament
final.tournament.value_counts()
# Frequency table for the  status
final["status"].value_counts()
 # plotting for the pie chart for status
 pie = final.status.value_counts().plot(kind="pie",autopct= "%1.1f%%")
 fig = plt.gcf()
 fig.set_size_inches(10,8)
 plt.axis('equal')
 plt.title('pie chart for status')
 plt.legend()
 plt.show()
# plotting bar graph for the tournament
plt.figure(figsize=(15,12),dpi = 100)
sns.countplot(x="tournament",data=final)
plt.xticks(rotation=90)
plt.show()<jupyter_output><empty_output><jupyter_text>Numerical variables<jupyter_code># checking for  summary statistics for numerical columns
final.describe().T
# Plotting bar graph for home score
import warnings
warnings.filterwarnings("ignore")
sns.boxplot(final.home_score,showmeans=True);
# Plotting bar graph for away score
sns.boxplot(final.away_score,showmeans=True)
plt.show()
# Plotting  pairplot
plt.figure(figsize=(10,8),dpi=100)
sns.pairplot(final[['home_score','away_score', 'home_team_rank', 'away_team_rank', 'competition']])
plt.show()
# plotting bar graph for  away score and away team
plt.figure(figsize=(20,16),dpi=200)
sns.barplot(x=final.away_score,y=final.away_team_rank,hue=final.status)
plt.show()
# plotting heat map to show correlation of varibles
plt.figure(figsize=(10,8),dpi=100)
sns.heatmap(final.corr(),annot=True,cmap="viridis")
plt.show()<jupyter_output><empty_output><jupyter_text>Checking for multi-collinearity<jupyter_code># plotting heatmap
corr = final.corr()
plt.figure(figsize=(12,8),dpi=150)
sns.heatmap(corr,annot=True,cmap= 'viridis')
plt.show()
# checking for multicollinearity
vif = pd.DataFrame(np.linalg.inv(corr.values), index = corr.index, columns = corr.columns)
vif
#sns.heatmap(vif,cmap="viridis",annot=True)<jupyter_output><empty_output><jupyter_text>The was no variable with high multi-collinearity  Approach 1 : Polynomial Regression<jupyter_code># dropping tournament column
final = final.drop('tournament',axis=1)
final.head(1)
# Dropping home_team and away team
final = final.drop(columns=['home_team','away_team'])
final.head(1)
final.head(1)
#  converting all categorical columns to numerical
final1 = pd.get_dummies(final,drop_first=True)
# Dropping date column
final1 = final1.drop('date',axis=1)
# separating our label from features
X = final1.drop('home_score',axis=1)
y = final1['home_score']
# importing polynomial 
from sklearn.preprocessing import PolynomialFeatures
polynomial_converter = PolynomialFeatures(degree=2,include_bias=False)
poly_features = polynomial_converter.fit_transform(X)
# Importing train text split
from sklearn.model_selection import train_test_split
# spliting train and text
X_train, X_test, y_train, y_test = train_test_split(poly_features, y, test_size=0.3, random_state=101)
# importing linear regression
from sklearn.linear_model import LinearRegression
# creating linear regression
model = LinearRegression(fit_intercept=True)
# fitting linear regression on our X_train and y_train
model.fit(X_train,y_train)
# Predicting the  y values
test_predictions = model.predict(X_test)
# Importing the metrics 
from sklearn.metrics import mean_absolute_error,mean_squared_error
# calculating the mean absolute error and root mean squared error
MAE = mean_absolute_error(y_test,test_predictions)
MSE = mean_squared_error(y_test,test_predictions)
RMSE = np.sqrt(MSE)
# mean absolute error
MAE
# root mean squared squared error
RMSE
# plotting the residual graph
residuals = test_predictions - y_test

# Plotting the prediction errors
plt.figure(figsize=(10,5))
plt.scatter(test_predictions, residuals, color='indigo')
#plt.title('Residual Plot')
plt.ylabel('residual')
plt.xlabel('fitted values')
plt.axhline(y= residuals.mean(), color='orange')
plt.show()<jupyter_output><empty_output><jupyter_text>It's also important to plot out residuals and check for normal distribution, this helps us understand if Linear Regression was a valid model choice. Our data is normal since its random<jupyter_code>#Bartlett's test
import scipy as sp
test_result, p_value = sp.stats.bartlett(test_predictions,residuals)
degree_of_freedom = len(test_predictions)-1
probability = 1 - p_value

critical_value = sp.stats.chi2.ppf(probability, degree_of_freedom)
print(critical_value)

if (test_result > critical_value):
  print('the variances are unequal, and the model should be reassessed')
else:
  print('the variances are homogeneous!')<jupyter_output>1437.5643443151755
the variances are homogeneous!<jupyter_text>MODEL 2 POLYNOMIAL  FOR AWAY SCORE<jupyter_code># Dropping date column
final = final.drop('date',axis=1)
final.head(1)
# converting category variable to numerical
final2 = pd.get_dummies(final,drop_first=True)
# creating features and label
X1 = final2.drop('away_score',axis=1)
y2 = final2['away_score']
# Creating polynomial features
from sklearn.preprocessing import PolynomialFeatures
polynomial_c = PolynomialFeatures(degree=2,include_bias=False)
poly_feature = polynomial_c.fit_transform(X1)
# train testing our dataset
X1_train, X1_test, y2_train, y2_test = train_test_split(poly_feature, y2, test_size=0.3, random_state=101)
# Creating a model
model1 = LinearRegression(fit_intercept=True)
# Fitting linear regression on our model
model1.fit(X1_train,y2_train)
# Predicting the z values
test_pred = model.predict(X1_test)
# Calculating the metrics of our model
MAE = mean_absolute_error(y_test,test_predictions)
MSE = mean_squared_error(y_test,test_predictions)
RMSE = np.sqrt(MSE)
# Mean absolute error of our model
MAE
# Root mean squared error of our model
RMSE<jupyter_output><empty_output><jupyter_text>LOGISTIC REGRESSION<jupyter_code># previewing the dataset
final.head(1)
# separating label from features
W = final.drop('status',axis=1)
z = final['status']
# separating train and test 
W_train, W_test, z_train, z_test = train_test_split(W, z, test_size=0.25, random_state=101)
# scaling and fitting  our dataset
scaler = StandardScaler()
scaled_W_train = scaler.fit_transform(W_train)
scaled_W_test = scaler.transform(W_test)<jupyter_output><empty_output><jupyter_text>Multi-Class Logistic Regression Model<jupyter_code># importing logistic regression
from sklearn.linear_model import LogisticRegression
# importing Gridsearch CV
from sklearn.model_selection import GridSearchCV
# Creating a model
log_model = LogisticRegression(solver='saga',multi_class="ovr",max_iter=5000)<jupyter_output><empty_output><jupyter_text>GridSearch for Best Hyper-ParametersMain parameter choices are regularization penalty choice and regularization C value.<jupyter_code># Penalty Type
penalty = ['l1', 'l2']

# Use logarithmically spaced C values (recommended in official docs)
C = np.logspace(0, 4, 10)
# creating grid search model
grid_model = GridSearchCV(log_model,param_grid={'C':C,'penalty':penalty})
# Fitting our model to the scaled dataset
grid_model.fit(scaled_W_train,z_train)
# checking the best params
grid_model.best_params_
# predicting z values
z_pred = grid_model.predict(scaled_W_test)
 # importing accuracy scaore and other metrics to measure our model
from sklearn.metrics import accuracy_score,confusion_matrix,classification_report,plot_confusion_matrix
# calculating the accuracy model
accuracy_score(z_test,z_pred)
# Creating confusion matrix
confusion_matrix(z_test,z_pred)
# Creating the confusion matrix
plot_confusion_matrix(grid_model,scaled_W_test,z_test)
# Classication report
print(classification_report(z_test,z_pred))<jupyter_output>precision    recall  f1-score   support

        Draw       1.00      1.00      1.00       240
        Loss       1.00      1.00      1.00       257
         Win       1.00      1.00      1.00       405

    accuracy                           1.00       902
   macro avg       1.00      1.00      1.00       902
weighted avg       1.00      1.00      1.00       902
{'fraction_non_alphanumeric': 0.0986441920117259, 'fraction_numerical': 0.018101868816416268, 'mean_word_length': 6.096203848153926, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '152104', 'license': 'Apache-2.0', 'n_tokens_mistral': 4484, 'n_tokens_neox': 4202, 'n_words': 1249, 'path': 'worldcup_pred.ipynb', 'repo_name': 'kelvinnjunge/world-cup-prediction', 'size': '13645'}