<jupyter_start><jupyter_text>**CO2 Emission Prediction**In this project we predict the cars CO2 emission based on 12 different features. We will investigate using both linear and polynomial regression models. The dataset used can be downloaded from [Kaggle](https://www.kaggle.com/sarita19/fuel-consumption). Load the libraries<jupyter_code>import pandas as pd
from scipy import stats
from sklearn.linear_model import LinearRegression
import seaborn as sns
from sklearn.metrics import mean_squared_error
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
import numpy as np<jupyter_output><empty_output><jupyter_text>**Load The Dataset**<jupyter_code>df = pd.read_csv('FuelConsumption.csv')
df.head(2)<jupyter_output><empty_output><jupyter_text>**Dataset Analysis**<jupyter_code>df.shape
df.dtypes
# check if any null values in the dataset
df.isnull().sum()
# just quick look at the distribution of the columns values
fig = df.hist(figsize=(120,120),xlabelsize =50,ylabelsize =50)
[x.title.set_size(50) for x in fig.ravel()]
plt.show()
# extract the correlation in the dataset to see what are the features that correalte with the co2 emission
df_corr = df.corr()
plt.figure(figsize=(20,15))
sns.heatmap(df_corr,annot=True)<jupyter_output><empty_output><jupyter_text>From the previous corrleation matrix, we can notice two features are postive and negtaive correlated with the target feature and they are ['FUELCONSUMPTION_COMB_MPG', 'FUELCONSUMPTION_CITY']<jupyter_code># with 'FUELCONSUMPTION_COMB_MPG' the correlation coefficient is very close to -1--> negative correlation and p-value is verysmall = 0 --> there is signifgnace
# with 'FUELCONSUMPTION_CITY' the correlation coefficient is very close to 1--> postive correlation and p-value is verysmall = 0 --> there is signifgnace
r_pvalue1 = stats.pearsonr(df['FUELCONSUMPTION_COMB_MPG'], df['CO2EMISSIONS'])
r_pvalue2 = stats.pearsonr(df['FUELCONSUMPTION_CITY'], df['CO2EMISSIONS'])
             
print(f'FUELCONSUMPTION_COMB_MPG: the correlation coefficient is {r_pvalue1[0]} and p-value is {r_pvalue1[1]}')
print(f'FUELCONSUMPTION_CITY: the correlation coefficient is {r_pvalue2[0]} and p-value is {r_pvalue2[1]}')<jupyter_output>FUELCONSUMPTION_COMB_MPG: the correlation coefficient is -0.9063942295226133 and p-value is 0.0
FUELCONSUMPTION_CITY: the correlation coefficient is 0.8980385119353924 and p-value is 0.0<jupyter_text>Linear Regression **Using: FUELCONSUMPTION_COMB_MPG**<jupyter_code>x = df[['FUELCONSUMPTION_COMB_MPG']]
y = df['CO2EMISSIONS']
sns.regplot(x,y)
plt.show()
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3, random_state =9)
SLR = LinearRegression()
SLR.fit(x_train,y_train)
SLR.coef_, SLR.intercept_
y_hat = SLR.predict(x_test)
SLR.score(x_test,y_test)
# or
from sklearn.metrics import r2_score
r2_score(y_test , y_hat)
mean_squared_error(y_test, y_hat)
sns.residplot(x_test, y_test)
plt.show()
# just to draw the regresion line, although we drew it already before
xx=np.arange(10, 60, 0.1)
plt.scatter(x_train, y_train,  color='blue')
plt.plot(xx, SLR.coef_[0]*xx + SLR.intercept_, '-r')
plt.xlabel("Engine size")
plt.ylabel("Emission")<jupyter_output><empty_output><jupyter_text>**Using: FUELCONSUMPTION_CITY**<jupyter_code>x = df[['FUELCONSUMPTION_CITY']]
y = df['CO2EMISSIONS']
sns.regplot(x,y)
plt.show()
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3, random_state =9)
SLR = LinearRegression()
SLR.fit(x_train,y_train)
SLR.score(x_test,y_test)
mean_squared_error(y_test, y_hat)
sns.residplot(x_test, y_test)
plt.show()<jupyter_output>/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variables as keyword args: x, y. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.
  FutureWarning<jupyter_text>**Using both FUELCONSUMPTION_COMB_MPG and FUELCONSUMPTION_CITY**<jupyter_code>x = df[['FUELCONSUMPTION_COMB_MPG','FUELCONSUMPTION_CITY']]
y = df['CO2EMISSIONS']
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3, random_state =9)
SLR = LinearRegression()
SLR.fit(x_train,y_train)
SLR.score(x_test,y_test)
mean_squared_error(y_test, y_hat)<jupyter_output><empty_output><jupyter_text>Polynomial Linear RegressionAs we noticed from the plots in the previous section, it seems the FUELCONSUMPTION_COMB_MPG feature doesnt have linear relationship with the target feature so let us try the polynomial regresion.<jupyter_code>from sklearn.preprocessing import PolynomialFeatures
from matplotlib import pyplot as plt
import numpy as np

x = df[['FUELCONSUMPTION_COMB_MPG']]
y = df['CO2EMISSIONS']
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.3, random_state =9)
pr = PolynomialFeatures(degree=2)
x_train_pr = pr.fit_transform(x_train)
x_test_pr = pr.fit_transform(x_test)
poly = LinearRegression()
poly.fit(x_train_pr, y_train)
yp_hat = poly.predict(x_test_pr)
poly.score(x_test_pr,yp_hat )
# or (r2_score  might be better)
from sklearn.metrics import r2_score
r2_score(y_test , yp_hat)
mean_squared_error(y_test, yp_hat)
poly.coef_
poly.intercept_
# function to plot the polynomial relationship
def PollyPlot(xtrain, xtest, y_train, y_test, lr,poly_transform):
    width = 12
    height = 10
    plt.figure(figsize=(width, height))
    
    
    #training data 
    #testing data 
    # lr:  linear regression object 
    #poly_transform:  polynomial transformation object 
 
    xmax=max([xtrain.values.max(), xtest.values.max()])

    xmin=min([xtrain.values.min(), xtest.values.min()])
# we create a new series of values to test the model prediction over the min-max of the data we have
    x=np.arange(xmin, xmax, 0.1)
    y_hat = lr.predict(poly_transform.fit_transform(x.reshape(-1, 1)))

    plt.plot(xtrain, y_train, 'ro', label='Training Data')
    plt.plot(xtest, y_test, 'go', label='Test Data')
    plt.plot(x, y_hat, label='Predicted Function')
    # plt.ylim([-10000, 60000])
    plt.ylabel('Price')
    plt.legend()
PollyPlot(x_train, x_test, y_train, y_test, poly,pr)

# Plot the distribution plot between the actual testset and the predicted testset
ax1 = sns.distplot(y_test, hist=False, label='orginal')
sns.distplot(yp_hat, hist=False, label='predicted', ax=ax1)
plt.show()<jupyter_output>/usr/local/lib/python3.6/dist-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `kdeplot` (an axes-level function for kernel density plots).
  warnings.warn(msg, FutureWarning)
/usr/local/lib/python3.6/dist-packages/seaborn/distributions.py:2551: FutureWarning: `distplot` is a deprecated function and will be removed in a future version. Please adapt your code to use either `displot` (a figure-level function with similar flexibility) or `kdeplot` (an axes-level function for kernel density plots).
  warnings.warn(msg, FutureWarning)<jupyter_text>we can notice how we get a little bit improvment in the results comparing to use 'FUELCONSUMPTION_COMB_MPG' as a feature with linear relationship. So let us try all the features with polynomial regression.<jupyter_code># determine what features are most corrleated in the dataset
df_corr = df.corr()
filt = (df_corr.loc['CO2EMISSIONS']>0.70) | (df_corr.loc['CO2EMISSIONS']<-0.70)
features = df_corr[filt].index.values.tolist()
features
df_MLR = df[features]
y = df_MLR['CO2EMISSIONS']
x = df_MLR.drop('CO2EMISSIONS',axis=1)
df_MLR.head(2)
x_train, x_test, y_train, y_test = train_test_split(x,y, test_size=0.3, random_state =9)
pr = PolynomialFeatures(degree=2)
x_train_pr = pr.fit_transform(x_train)
x_test_pr = pr.fit_transform(x_test)
poly = LinearRegression()
poly.fit(x_train_pr, y_train)
poly.coef_
poly.intercept_
yp_hat = poly.predict(x_test_pr)
poly.score(x_test_pr, y_test)
# or
r2_score(y_test, yp_hat)
mean_squared_error(y_test, yp_hat)
# Plot the distribution plot between the actual testset and the predicted testset

ax1 = sns.distplot(y_test, hist=False, label='orginal')
sns.distplot(yp_hat, hist=False, label='predicted', ax=ax1)
plt.show()<jupyter_output><empty_output><jupyter_text>Results AnalysisIt seems many features are correlated but without linear relationship with the target feature and therefore, the polynomial regression provides us better results. Advantages & Disadvantages of using Polynomial Regression:**Advantages***   Polynomial provides the best approximation of the relationship between the dependent and independent features (i.e. the input features and the target features).*   A Broad range of function can be fit under it.*   Polynomial basically fits a wide range of curvature.**Disadvantages** * The presence of one or two outliers in the data can seriously affect the results of the nonlinear analysis.SO, these are too sensitive to the outliers.* In addition, there are unfortunately fewer model validation tools for the detection of outliers in nonlinear regression than there are for linear regression. Referencehttps://towardsdatascience.com/introduction-to-linear-regression-and-polynomial-regression-f8adc96f31cb<jupyter_code><jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10145860145860146, 'fraction_numerical': 0.018232518232518233, 'mean_word_length': 6.430278884462151, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 43, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '632160', 'license': 'MIT', 'n_tokens_mistral': 3418, 'n_tokens_neox': 3261, 'n_words': 863, 'path': 'Regresion_CO2_Emission.ipynb', 'repo_name': 'MohamadALissa/CO2_Cars_Prediction', 'size': '9324'}