<jupyter_start><jupyter_text>Defining the question  **a) Specifying the Question**> You have been recruited as a football analyst in a company - Mchezopesa Ltd and tasked to accomplish the task below.A prediction result of a game between team 1  and team 2, based on who's home and who's away, and on whether or not the game is friendly (include rank in your training). You have two possible approaches (polynomial and logistic approaches) given the datasets that will be provided  **b) Defining the metrics for success**> The objective for this data includes investigating whether or ot the game is friendly  while making the opredictions  **c) Understanding the context**> You have been recruited as a football analyst in a company - Mchezopesa Ltd and tasked to accomplish the task below. ---- Predict the result of a game between team 1 and team 2, based on who's home and who's away, and on whether or not the game is friendly (include rank in your training).  **d) Recording the Experimental Design**1. reading in the datasets2. checking the data for duplicates and missinng data3. dropping of duplicates and removal filling in of missing data values with zero4. Finding  and dealing with outliers, anomalies, and missing data within the dataset.5. Performing of EDA recording of observations.6. Implementing the solution by performing the respective analysis i.e. factor analysis, principal component analysis, and discriminant analysis.8. Perform any necessary feature engineering 9. Check of multicollinearity10. Start building the model, Cross-validate the model11. Compute RMSE12. Create residual plots for your models, and assess their heteroscedasticity using Bartlett’s test13. Perform appropriate regressions on the data including your justification14. Challenge your solution by providing insights on how you can make improvements.  **e) Relevance of the data**> The data that was used for survey was obtained from an open datasource. this data is an aggregation by day of the number of goals scored and rankings of countries in the men's FIFA team. with it is a model that determines which team gets to be ranked where and how according to the years and goals.  **Checking the data**<jupyter_code># importing libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib
from matplotlib import pyplot as plt
%matplotlib inline
##
# reading datasets

data = pd.read_csv('fifa_ranking.csv')
data1 = pd.read_csv('results.csv')
data.rename(columns= {'country_full':'country'}, inplace = True)
# droppig unnecessary columns
to_drop = ['country_abrv', 'rank_date']
data.drop(to_drop, axis=1, inplace=True)
data.columns
data.shape
# checking for duplicates
data.duplicated().any()
# drop all duplicates
data.drop_duplicates(inplace=True)
data.duplicated().sum()
# checking for null values
data.isnull().any()
# checking for duplicates
data1.duplicated().any()
# checking for null values
data1.isnull().any()
#merging datasets
frames = [data, data1]
df = pd.concat(frames)
df
df.head()
df.isnull().any()
# finding shape of the dataset
df.shape
# reading columns
df.columns
# finding shape of the dataset
df.shape
#previewing dataset
df.head()
df.tail()
df.sample(7)
# checking datatypes
df.info()
# checking for duplicates
df.duplicated().any()
# drop all duplicates
df.drop_duplicates(inplace=True)
df.duplicated().sum()
# checking for null values
df.isnull().any()
# filling null values with zero
df.fillna(0, inplace=True)
# Dealing with the Missing Data
df.isnull().sum()
# dealing with outliers
# quantiles
Q1 = df.quantile(0.25)
Q3 = df.quantile(0.75)
IQR = Q3 - Q1

# Removing outliers based on the IQR range and stores the result in the data frame 'df_out'
# ---
# 
df1 = df[~((df < (Q1 - 1.5 * IQR)) | (df > (Q3 + 1.5 * IQR))).any(axis=1)]

# Printing the shape of our new dataset
# ---
# 
print(df1.shape)

# Printing the shape of our old dataset
# ---
#
print(df.shape)
# top football confederations
print('Total number of confederations: {0}'.format(df1['confederation'].nunique()))
print(df1['confederation'].value_counts().head(10))
print('--'*40)<jupyter_output>Total number of confederations: 7
0           26356
CAF          2054
UEFA         2035
AFC          1759
CONCACAF     1434
OFC           470
CONMEBOL      412
Name: confederation, dtype: int64
--------------------------------------------------------------------------------<jupyter_text>**EXPLORATORY DATA ANALYSIS**  **Univariate analysis**<jupyter_code>#Histogram of the ome scores of different countries
ax = sns.countplot(df1['home_score'], order = df1['home_score'].value_counts().index)
plt.title('Distribution of countries per their homescores')
## boxplot
sns.boxplot(df1['home_score'], showmeans=True)
sns.boxplot(df1['away_score'], showmeans=True)
df1['away_score'].value_counts().sort_index().plot.bar()
df1['home_score'].value_counts().sort_index().plot.bar()
print(df1.skew())
print(df1.kurt())
df1.columns
to_drop = ['rank_change',
       'cur_year_avg', 'cur_year_avg_weighted', 'last_year_avg',
       'last_year_avg_weighted', 'two_year_ago_avg', 'two_year_ago_weighted',
       'three_year_ago_avg', 'three_year_ago_weighted']
df1.drop(to_drop, axis=1, inplace=True)
df1.shape<jupyter_output><empty_output><jupyter_text>**Bivariate analysis**<jupyter_code># Ploting the bivariate summaries and recording our observations
# scatterplot
sns.pairplot(df1)
plt.show()
# Calculating the pearson coefficient
pearson_coeff = df1["home_score"].corr(df1["away_score"], method="pearson") 
print(pearson_coeff)

# Checking whether you have to define the pearson
coeff = df1["home_score"].corr(df1["away_score"])
print(coeff)
# heatmap
sns.heatmap(df1.corr(),annot=True)
plt.show()
df1.sample(6)
df1.info()
df1['home_score'].corr(df1['away_score'])
# Determine correlation between numeric features
plt.figure(figsize=(12,10))
sns.pairplot(df1[['home_score','away_score']])
plt.title('home_score vs away_score')<jupyter_output><empty_output><jupyter_text>**Multivariate analysis**  **Factor analysis**<jupyter_code># Installing factor analyzer 
!pip install factor_analyzer==0.2.3
from factor_analyzer.factor_analyzer import calculate_bartlett_sphericity
chi_square_value,p_value=calculate_bartlett_sphericity(df)
chi_square_value, p_value

#The p vlaue is 0 the observed matrix is not an identity matrix.<jupyter_output>Requirement already satisfied: factor_analyzer==0.2.3 in /usr/local/lib/python3.6/dist-packages (0.2.3)
Requirement already satisfied: pandas in /usr/local/lib/python3.6/dist-packages (from factor_analyzer==0.2.3) (1.0.5)
Requirement already satisfied: scipy in /usr/local/lib/python3.6/dist-packages (from factor_analyzer==0.2.3) (1.4.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from factor_analyzer==0.2.3) (1.18.5)
Requirement already satisfied: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from pandas->factor_analyzer==0.2.3) (2.8.1)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->factor_analyzer==0.2.3) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.6/dist-packages (from python-dateutil>=2.6.1->pandas->factor_analyzer==0.2.3) (1.15.0)<jupyter_text>**Principal component analysis**<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaler.fit(df1[['home_score','away_score']])
scaled_data = scaler.transform(df1[['home_score','away_score']])
scaled_data = pd.DataFrame(scaled_data)
df2 = scaled_data.fillna(0)
df2.head()
from sklearn.decomposition import PCA
logmodel = PCA(n_components=2)
logmodel.fit(df2)
pca_x = logmodel.transform(df2)
pca_x.shape
df3 = pd.DataFrame(logmodel.components_, columns = ['home_score','away_score'])
plt.figure(figsize=(10,6))
sns.heatmap(df3)
plt.figure(figsize=(10,6))
ty=sns.scatterplot(pca_x[:,0], pca_x[:,1])
sns.despine(left=True)
ty.set_title('PCA Results')
ty.set_ylabel('Second Principle Component ')
ty.set_xlabel('First Principle Component ')
pca_x<jupyter_output><empty_output><jupyter_text>**Polynomial Regression Analysis**<jupyter_code>#Import libaries
import datetime
import operator
import matplotlib.pyplot as plt
from ipywidgets import interact
from IPython.display import Markdown as md
from datetime import date, timedelta, datetime
from numpy import sqrt
from pandas import read_csv, to_datetime
from prettytable import PrettyTable
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.preprocessing import PolynomialFeatures
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split

# Import LinearRegression method from sklearn linear_model library
from sklearn.linear_model import LinearRegression
df1.shape
df1.dtypes
df5 = df1
df5.head()
df5.columns
# droppig unnecessary columns
to_drop = ['country', 'total_points', 'previous_points', 'rank_change',
       'cur_year_avg', 'cur_year_avg_weighted', 'last_year_avg',
       'last_year_avg_weighted', 'two_year_ago_avg', 'two_year_ago_weighted',
       'three_year_ago_avg', 'three_year_ago_weighted', 'confederation',
       'date', 
       'tournament', 'city', 'neutral']
df5.drop(to_drop, axis=1, inplace=True)
print(df5)

#Get the 2 and the last column from the dataset 
X = df5.iloc[:, 3:4].values
y = df5.iloc[:, 4].values
print(X)
print(y)
# Split the dataset into train and test sets
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=0)
# Fit Linear Regression model to the dataset(this is optional. We are doing this for the sole purpose of comparing the linear regression model to the polynomial regression model)
from sklearn.linear_model import LinearRegression
lin_reg = LinearRegression()
lin_reg.fit(X, y)
from sklearn.preprocessing import PolynomialFeatures
poly_reg2=PolynomialFeatures(degree=2)
X_poly=poly_reg2.fit_transform(X)
lin_reg_2=LinearRegression()
lin_reg_2.fit(X_poly,y)
poly_reg3=PolynomialFeatures(degree=3)
X_poly3=poly_reg3.fit_transform(X)
lin_reg_3=LinearRegression()
lin_reg_3.fit(X_poly3,y)
# Visualize the Linear Regression results
plt.scatter(X, y, color='blue')
plt.plot(X, lin_reg.predict(X), color='red')
plt.title('home score vs away_score')
plt.xlabel('away_score')
plt.ylabel('home_score')
plt.show()

# Visualize the polynomial Regression results
plt.scatter(X,y,color='red')
plt.plot(X,lin_reg_2.predict(poly_reg2.fit_transform(X)),color='blue')
plt.plot(X,lin_reg_3.predict(poly_reg3.fit_transform(X)),color='green')
plt.title('home score vs away_score')
plt.xlabel('away_score')
plt.ylabel('home_score')
plt.show()
X_grid=np.arange(min(X),max(X),0.1) # This will give us a vector.We will have to convert this into a matrix 
X_grid=X_grid.reshape((len(X_grid),1))
plt.scatter(X,y,color='red')
plt.plot(X_grid,lin_reg_3.predict(poly_reg3.fit_transform(X_grid)),color='blue')
#plt.plot(X,lin_reg_3.predict(poly_reg3.fit_transform(X)),color='green')
plt.title('home score vs away_score')
plt.xlabel('away_score')
plt.ylabel('home_score')
plt.show()
# droppig unnecessary columns
to_drop = ['home_team', 'away_team']
df5.drop(to_drop, axis=1, inplace=True)
# Model 1: Predict how many goals the home team scores.
lin_reg.predict([[6.5]])
# multicolinearity
correlations = df5.corr()
correlations

# the table below shows how the variables relate with each other. 
# there is very minimal correlation between the scores at home and away
pd.DataFrame(np.linalg.inv(correlations.values), index = correlations.index, columns=correlations.columns)

print(revised.values)
df5.rename(columns= {'home_score':'homescore', 'away_score':'awayscore'}, inplace = True)
# residual plots and heteroskedasticity testing
from sklearn import metrics
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

X = df5.values
y = df5['homescore'].values

X_train, X_test, homescore_train, homescore_test = train_test_split(X, y, test_size=0.4, random_state=0)

regressor = LinearRegression()
regressor.fit(X_train, homescore_train)

# This is our prediction for admission based on our model
homescore_predict = regressor.predict(X_test)

# We now create the residual by substracting the test value from the predicted 
# value for each row in our dataset

residuals = np.subtract(homescore_predict, homescore_test)

# Let's describe our residual:
pd.DataFrame(residuals).describe()

residuals.mean()
# residual plot
import matplotlib.pyplot as plt

plt.scatter(homescore_predict, residuals, color='black')
plt.ylabel('residual')
plt.xlabel('fitted values')
plt.axhline(y= residuals.mean(), color='red', linewidth=1)
plt.show()
import scipy as sp

test_result, p_value = sp.stats.bartlett(homescore_predict, residuals)

# To interpret the results we must also compute a critical value of the chi squared distribution
degree_of_freedom = len(homescore_predict)-1
probability = 1 - p_value

critical_value = sp.stats.chi2.ppf(probability, degree_of_freedom)
print(critical_value)

# If the test_result is greater than the critical value, then we reject our null
# hypothesis. This would mean that there are patterns to the variance of the data

# Otherwise, we can identify no patterns, and we accept the null hypothesis that 
# the variance is homogeneous across our data

if (test_result > critical_value):
  print('the variances are unequal, and the model should be reassessed')
else:
  print('the variances are homogeneous!')

# Training and Cross Validation
# ---
# 
from sklearn.ensemble import RandomForestClassifier
classifier = RandomForestClassifier(n_estimators=300, random_state=0)
from sklearn.model_selection import cross_val_score
all_accuracies = cross_val_score(estimator=classifier, X=X_train, y=y_train, cv=5)
# Printing the accuracies returned for five folds 
# by the cross_val_score method by calling print on all_accuracies

print(all_accuracies.mean())
# specify parameters and distributions to sample from
from scipy.stats import randint as sp_randint
param_dist = {"max_depth": [3, None],
              "max_features": sp_randint(1, 11),
              "min_samples_split": sp_randint(2, 11),
              "bootstrap": [True, False],
              "criterion": ["gini", "entropy"]}
#Instantiating RandomizedSearchCV object 
from sklearn.model_selection import RandomizedSearchCV 
random_sr = RandomizedSearchCV(classifier, param_dist, cv = 5) 
#Calling the fit method
random_sr.fit(X_train, y_train)
#Checking the parameters that return the highest accuracy

best_parameters = random_sr.best_params_
print(best_parameters)
# Finding the obtained accuracy

best_result = random_sr.best_score_
print(best_result)<jupyter_output>1.0<jupyter_text>**Logistics Regression Analysis**<jupyter_code># dealing with outliers
# quantiles
Q1 = df.quantile(0.25)
Q3 = df.quantile(0.75)
IQR = Q3 - Q1

# Removing outliers based on the IQR range and stores the result in the data frame 'df_out'
# ---
# 
df6 = df[~((df < (Q1 - 1.5 * IQR)) | (df > (Q3 + 1.5 * IQR))).any(axis=1)]

# Printing the shape of our new dataset
# ---
# 
print(df6.shape)

# Printing the shape of our old dataset
# ---
#
print(df.shape)
df6.columns
import pandas as pd
import seaborn as sb
sb.countplot(x='home_score',data=df6, palette='hls')

# Checking for independence between features¶
# 
sb.heatmap(df6.corr())  
# splitting our dataset
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = .3, random_state=25)
# fitting our model
from sklearn.linear_model import LogisticRegression

LogReg = LogisticRegression()
LogReg.fit(X_train, y_train)
# using our model to make a prediction
y_pred = LogReg.predict(X_test)
y_pred
# Evaluating the model
#
from sklearn.metrics import confusion_matrix
confusion_matrix = confusion_matrix(y_test, y_pred)
confusion_matrix<jupyter_output><empty_output><jupyter_text>Logistic Regression5.0 Pts**Excellent**The students exhaustively perform Logistic regression by performing feature engineering where necessary, Training an optimal classifier, Interpret the result obtained and make a suitable prediction. The students should also challenge their solution by providing insights on how you can make improvements.  **Discriminant Analysis**<jupyter_code>from sklearn.model_selection import StratifiedKFold
from sklearn.metrics import roc_auc_score
from sklearn.preprocessing import StandardScaler
from sklearn.feature_selection import VarianceThreshold

from sklearn.discriminant_analysis import QuadraticDiscriminantAnalysis

from tqdm import tqdm_notebook

import warnings
warnings.filterwarnings('ignore')
df4 = df1.fillna(0)
df4.shape
X = df4.iloc[:, 0:4].values
y = df4.iloc[:2, ].values

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_test = sc.transform(X_test)
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA
lda = LDA(n_components=1)
X_train = lda.fit_transform(X_train, y)
X_test = lda.transform(X_test)
from sklearn.ensemble import RandomForestClassifier

classifier = RandomForestClassifier(max_depth=2, random_state=0)
classifier.fit(X_train, y_train)
y_pred = classifier.predict(X_test)
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score

cm = confusion_matrix(y_test, y_pred)
print(cm)
print('Accuracy' + str(accuracy_score(y_test, y_pred)))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10788099023393141, 'fraction_numerical': 0.018396547808312513, 'mean_word_length': 5.909768536681051, 'pattern_counts': {'":': 5, '<': 32, '<?xml version=': 0, '>': 45, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '466473', 'license': 'MIT', 'n_tokens_mistral': 5953, 'n_tokens_neox': 5671, 'n_words': 1629, 'path': 'IPWK6_CORE_LINEAR_REGRESSION_ELIZABETH_JOSEPHINE.ipynb', 'repo_name': 'alfafimel/IPWK6-CORE', 'size': '17612'}