<jupyter_start><jupyter_text>Learning Objectives Include:        * Simple and Multiple Linear Regression    * Model Evaluation using Visualization    * Polynomial Regression and Pipelines    * R-Squared and MSE for In-Sample Evaluation    * Prediction and Decision Making    Question:-       How can we determine a fair value for a used car?<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import scipy.stats as stats
from google.colab import drive
drive.mount('/content/gdrive')
file_data = ''
with open('/content/gdrive/My Drive/Colab_Notebooks/IBM_cognitive_class_learning/adjusted_df.csv' ) as f:
    file_data = f.read()
import sys
if sys.version_info[0] < 3: 
    from StringIO import StringIO
else:
    from io import StringIO

TESTDATA = StringIO(file_data)

adjusted_df = pd.read_csv(TESTDATA, sep=",")

adjusted_df.head()<jupyter_output><empty_output><jupyter_text>The first column is unnecessary and should be dropped. Let's do so now<jupyter_code>adjusted_df = adjusted_df.iloc[:, 1:]
adjusted_df.head()
adjusted_df.shape<jupyter_output><empty_output><jupyter_text>Let's check the number of missing values in the data set<jupyter_code>adjusted_df.columns
adjusted_df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 201 entries, 0 to 200
Data columns (total 27 columns):
symboling            201 non-null int64
normalized-losses    201 non-null object
make                 201 non-null object
fuel-type            201 non-null object
aspiration           201 non-null object
num-of-doors         201 non-null object
body-style           201 non-null object
drive-wheels         201 non-null object
engine-location      201 non-null object
wheel-base           201 non-null float64
length               201 non-null float64
width                201 non-null float64
height               201 non-null float64
curb-weight          201 non-null int64
engine-type          201 non-null object
num-of-cylinders     201 non-null object
engine-size          201 non-null int64
fuel-system          201 non-null object
bore                 201 non-null float64
stroke               201 non-null float64
compression-ratio    201 non-null float64
horsepower           201 non-n[...]<jupyter_text>We need to do some data wrangling and distribution analysis<jupyter_code># Let's clean the num-of-doors column or feature.

# Lets view the unque elements in this column and count of each unique element

adjusted_df.groupby('num-of-doors').size()
# Next lets replace missing values to the mode, which is clearly four door cars

adjusted_df['num-of-doors'].replace('?','four', inplace=True)

# Next let's see the update
adjusted_df.groupby('num-of-doors').size()<jupyter_output><empty_output><jupyter_text>Let's investigate the relationship between num-of-doors and price<jupyter_code>plt.figure(figsize=(10,8))
sns.set_style('ticks')

plt.title('Boxplot showing relations between Num-of-Doors and Price', fontsize=14)
sns.boxplot('num-of-doors', 'price', data=adjusted_df)
plt.xticks(color='aqua')
plt.yticks(color='aqua')
#plt.xlabel(color='aqua')
#plt.ylabel(color='aqua')

plt.show()
# Let's find the average price for two and four door vehicles

adjusted_df[['num-of-doors', 'price']].groupby('num-of-doors').mean()<jupyter_output><empty_output><jupyter_text>_**From the box plot above and the groupby() method above, we can see that there is little difference between a 4 door and 2 door vehicle. So the num-of-doors categorical variable is not a good indicator or predictor of the price of cars**_  Let's investigate the relationship between num-of-cylinders and price<jupyter_code># first lets group the column and see the types and count of unique elements

adjusted_df.groupby('num-of-cylinders').size()
# Let's create some boxplot visualization

plt.figure(figsize=(10,8))
plt.title('Boxplot showing relations between Num-of-Cylinders and Price')
sns.boxplot('num-of-cylinders', 'price', data=adjusted_df)
plt.show()
adjusted_df[['num-of-cylinders','price']].groupby('num-of-cylinders').mean()<jupyter_output><empty_output><jupyter_text>**From the box plot above and the groupby() method above, we can see that there is significant difference between the various num-of-cylinders variable, except 5 and 6 cylinders which are almost identical.**  **So this categorical variable is possibly a good indicator or predictor of the price of cars**_ What is a Model?**A model can be thought as a mathematical equation used to Predict a Value, given One or More other Values,**  **Relating one or more independent variables or features to dependent variables.**_Usually the more relevant data you have, the more accurate your model is_ Linear Regression and Multiple Linear Regression: **Linear regression:** Refers to the use of one independent variable to make a prediction.  **Multiple Linear Regression:** Refers to the use of multiple independent variables to make a prediction.  Simple Linear Regression:  $y = b_0 + b_1x$  $y$ = **The dependent variable or target variable.**  $x$ = **The independent variable or feature or the predictor.**  $b_0$ = **The intercept or where the regression line meets the target variable y.**  $b_1$ = **The slope.** Process Summary for Simple Linear Regression  * **We have a set of Training Points**.  * **We use these Training Points to train or Fit the Model and get Parameters($b_0,b_1$).**  * **We then use these parameters in the Model**  * **We now have a model... and we put a hat on the y($y^$) to show the model is an estimate.**  * **We can use this model to predict values that we have never seen.**  * _**Don't forget that our model is not always correct... sometimes due to the noise in the data, or other factors.**_ Fitting a simple linear regression Model in Python * **Import Linear_model from scikit-learn.**  *(from sklearn.linear_model import LinearRegression)*  * **Create a Linear_regression object using the constructor.**  _lm=LinearRegression()_  * **We define the Predictr variable($X$) and Target variable($Y$).**  * **Then use the method .fit() to fit the model and find parameters $b_0,b_1$.**  _lm.fit(X,Y)_  * **We can obtain a prediction using the .predict() method.**  _Yhat = lm.predict(X)_  * **The output is an array, which has the same number of samples as the input $X$.**  * **The intercept $b_0$ is an attribute of the object** _lm_**. It is the value of $Y$ when $X$ is $0$.***we can view the intercept by- lm.intercept_*  * **The slope $b_1$ is also an attribute of the object** _lm_**. It is the value with which $Y$ changes due to 1 unit change in $X$**  *we can view the slope by- lm.coef_* let's create a simple linear regression model using highway-mpg to predict price<jupyter_code># Step one_ importing scikit learn

from sklearn.linear_model import LinearRegression
# Step two_ creating a linear_regression object 

lm = LinearRegression()
lm
# Step threeA_ Defining X and Y, both or X must be in 2D or like a dataframe, not a series.

X = adjusted_df[['highway-mpg']]
Y = adjusted_df['price']
# Step threeB_ lets fit the model to find our intercept(b0) and slope(b1)

lm.fit(X,Y)
# Let's predict Y based on X

Yhat = lm.predict(X)
Yhat
# To view our intercept

intercept = lm.intercept_
intercept
# To view the slope

slope = lm.coef_
slope<jupyter_output><empty_output><jupyter_text>The relationship between Highway-mpg and Price is given by;  $y = b_0 + b_1x$ This is same as yhat(Price)  =  intercept  +  slope * Highway-mpg  $Price(Yhat) = 38423.3059 + (-821.7334)* Highway-mpg(X)$<jupyter_code><jupyter_output><empty_output><jupyter_text>Multiple Linear Regression **This method is used to explain the relationship between:-**      **One continous Target variable(Y) and,**    **Two or more Independent or Predictor variables(X).** Fitting a Multiple Linear Regression Model in Python  **1. Let's extract our multiple variables, for example:- horsepower, curb-weight, engine-size, highway-mpg.**<jupyter_code>predictors = adjusted_df[['horsepower','curb-weight','engine-size','highway-mpg']]
target = adjusted_df[['price']]
predictors.head()
target.head()<jupyter_output><empty_output><jupyter_text>**2. Let's train our model as before using the predictor and target variables.**<jupyter_code>dm = LinearRegression()
dm.fit(predictors, target)<jupyter_output><empty_output><jupyter_text>**3. We can also obtain a prediction using .predict() method.**<jupyter_code>y_hat = dm.predict(predictors)
y_hat[:10]<jupyter_output><empty_output><jupyter_text>**4. The intercept $b_0$ is an attribute of the object.**<jupyter_code>prd_intacpt = dm.intercept_ 
prd_intacpt<jupyter_output><empty_output><jupyter_text>**5. The slope $b_1$ is also an attribute of the object.**  **Note that the slope or coefficient of each variable is returned in a 2D array in multiple linear regression model.**<jupyter_code>prd_slope = dm.coef_
prd_slope<jupyter_output><empty_output><jupyter_text>Let's visualize the multiple linear regression equation, replacing the independent var names with actual names.   The Estimated Linear Model for Multiple Linear Regression is:  $y^$ $= b_0 + b_1x_1 + b_2x_2 + b_3x_3 + b_4x_4$.  _($y^$ is the predicted variable, $b_0$ is the intercept, $b_1$ is the coef or slope of var $x_1$, same as $b_2$$x_2$...$b_n$$x_n$.)_ _**which can be further wriiten as the following, in response to our four independent variables**_  **Price** $= -15851.38 + (53.79*horsepower) + (4.71*curb-weight) + (81.38*engine-size) + (36.93*highway-mpg)$ ><jupyter_code><jupyter_output><empty_output><jupyter_text>Model Evaluation using Visualization Residual Plot:  **This represents the error between the actual value. Examining the predicted value and actual value, we see a difference.**  **We obtain that value by subtracting the predicted value from the actual target value.  We then plot that value on the vertical axis. With the dependent(target) variable on the horizontal axis.**  **Similarly, for the second variable we repeat the process above.**  Indication:       * _we expect to see the results to have zero mean, distributed evenly along the x axis with similar variance._  * _There should be no curvature and the variance should be uniform all across, not rising or falling at some point_   * _When we look at a regression plot, the residual is the distance from the data point to the fitted regression line._   So what is a Residual plot?A residual plot is a graph that shows the residuals on the vertical y-axis and the independent variable on the horizontal x-axis.What do we pay attention to when looking at a residual plot?We look at the spread of the residuals:- If the points in a residual plot are randomly spread out around the x-axis, then a linear model is appropriate for the data. Why is that? Randomly spread out residuals means that the variance is constant, and thus the linear model is a good fit for this data.Residual plots are ideal for visualizing the spread of the data in Simple Linear Regression Models Plotting a Residual Plot<jupyter_code>plt.figure(figsize=(12,10))
plt.title('Residplot for Highway-mpg and Price')
sns.residplot(adjusted_df['highway-mpg'], target)
plt.show()<jupyter_output><empty_output><jupyter_text>Distribution Plots:         **Distplots count the predicted values versus the actual values.**     **Distplots are extremely useful for visualizing models with morethan one independent variable or feature**<jupyter_code>plt.figure(figsize=(10,8))
plt.title('Distplot for Single Linear Regression Model-lm: Highway-mpg')
plt.ylabel('Highway-mpg')
ax1 = sns.distplot(adjusted_df['price'], hist=False, color='r', label='Actual Value')
sns.distplot(Yhat, hist=False, color='b', ax = ax1, label='Fitted Value')<jupyter_output><empty_output><jupyter_text>_**The Distplot above for SLR has no values for car price from 30000 - 50000. This suggests that either a Non-linear model may be suitable for the model or we simply need more data to make the model more accurate.**_<jupyter_code>plt.figure(figsize=(10,8))
plt.title('SLR Model Scatter plot- Highway-mpg and Price')
plt.scatter(X,Y, color = 'r')
plt.plot(X, Yhat, color='b')
plt.show()
#First set the size,title and labels
plt.figure(figsize=(10,8))
plt.title('Regplot for Single Linear Regression Model- Highway mpg')
plt.xlabel('Highway-mpg')

# next let's convert Y from a dataframe to a pandas series so we can fit it in the regplot
Yh = Y.squeeze()

# Then plot using regplot
sns.regplot(Yhat, Yh , line_kws={'color':'red', 'linewidth':2.5})

plt.ylim(0,)
plt.figure(figsize=(10,8))
plt.title('Distplot for MLR Model-dm:- horsepower, curb-weight, engine-size, highway-mpg')
ax1 = sns.distplot(target, hist=False, color='r', label='Actual Value')
sns.distplot(y_hat, hist=False, color='b', ax = ax1, label='Fitted Value')
#First set the size,
plt.figure(figsize=(10,8))
plt.title('Regplot for MLR Model:- horsepower, curb-weight, engine-size, highway-mpg')

# next let's convert target from a dataframe to a pandas series, so we can fit it in the regplot
Yh = target.squeeze()

# Then plot using regplot
sns.regplot(y_hat, Yh, color='r', line_kws={'color':'navy','linewidth':2.5})

plt.ylim(0,)<jupyter_output><empty_output><jupyter_text>Polynomial Regression and Pipelines  What do we do when a linear model is not the best fit for our data?  We transform our data into a Polynomial, then use linear regression to fit the parameter.Pipelines are a way to simplify our codePolynomial regression is a special case of linear regresion. This method is better fit for describing curvilinear relations. A Curvilinear relationship is what we get by squaring or setting higher order terms for the predictor variables, transforming the data. Common types of Polynomial Regression Models:    1. Quadratic model:       $y^$  =  $b_0 + b_1x^2 + b_2x^2.$        _This means the predictor variable is Squared in the model. This is a Second order Polynomial Regression._         2. Cubic model:       $y^$  =  $b_0 + b_1x^2 + b_2x^2 + b_3x^3.$        _This means the predictor variable is Cubed in the model. This is a Third Order Polynomial Regression._3. Higher Order:       $y^$  =  $b_0 + b_1x^2 + b_2x^2 + b_3x^3 +...$        _There also exists Higher Order Polynomial Regression, when a good fit hasn't been achieved by 2nd or 3rd Order ._        **The degree of the regression makes a big difference and can result in a better fit if we pick the right value.**  **In all cases, the relationship between the variable and the parameters or coefficients is always linear.** PLR Model for Highway-mpg and Price Approach_One<jupyter_code>from sklearn.preprocessing import PolynomialFeatures
poly = PolynomialFeatures(degree=6)
poly
# remember we defined highway-mpg as X before, so we just use it here

X_poly = poly.fit_transform(X)
# Let's now create a new Linear Regression Model

lm3 = LinearRegression()
# let's train the new linear model with the polynomial features of X the predictor variable and Y the target

lm3.fit(X_poly, Y)
result = lm3.predict(X_poly)<jupyter_output><empty_output><jupyter_text>Visualizing PLR model Distplot of Highway-mpg and Price<jupyter_code>plt.figure(figsize=(10,8))
plt.title('PLR Distplot for SLR Model-lm3: Highway-mpg and Price')
ax1 = sns.distplot(Y, hist=False, color='r', label='Target')
sns.distplot(result, hist=False, color='b', label='Fitted', ax=ax1)
plt.show()
plt.figure(figsize=(10,8))
plt.title('Regplot showing PLR for SLR Model highway-mpg and Price')
Yh = Y.squeeze()
sns.regplot(result,Yh,color='b')
plt.show()
lm3.score(X_poly,Y)<jupyter_output><empty_output><jupyter_text>Approach_Two<jupyter_code># First, let's look at the highway-mpg stats once again

adjusted_df[['highway-mpg']].describe()
# Let's set the poly method

def plotpoly(model, predictors,target, name):
    x_new = np.linspace(15,55,100)
    y_new = model(x_new)
    
    plt.figure(figsize=(8,6))
    plt.plot(predictors, target,'.', x_new, y_new,'--')
    plt.title("Polynomial Fit for Highway-mpg and Price")
    ax = plt.gca()
    ax.set_facecolor((0.890, 0.890, 0.890))
    #fig = plt.gcf()
    plt.xlabel(name)
    plt.ylabel('Price of Cars')
    
    plt.show()
    plt.close()
# Let's set the variables

X = adjusted_df['highway-mpg']
Y = adjusted_df['price']<jupyter_output><empty_output><jupyter_text>Let's fit the polynomial using the function polyfit, then use the function poly1d to display the polynomial function.<jupyter_code># Here we use a polynomial of the third order- cubic model.

f = np.polyfit(X,Y,6)
p = np.poly1d(f)

print(p)
# Let's plot the function

plotpoly(p, X, Y, 'Highway-mpg')<jupyter_output><empty_output><jupyter_text>PLR for MLR<jupyter_code># First let's create a polynomial feature model

Mpoly = PolynomialFeatures(degree=4)
Mpoly
# lets confirm the shape of the predictor or independent variable before polynomially fitting it

predictors.shape
# Let's fit it and confirm the shape

predict_fit= Mpoly.fit_transform(predictors)
predict_fit.shape
lm4 = LinearRegression()
lm4.fit(predict_fit, target)
mlr_result = lm4.predict(predict_fit)<jupyter_output><empty_output><jupyter_text>PLR for MLR Visualization<jupyter_code>plt.figure(figsize=(10,8))
plt.title('Distplot showing PLR for MLR Model- lm4')
ax1 = sns.distplot(target, hist=False, color='r', label="Target")
sns.distplot(mlr_result, hist=False, color='b', label='Fitted', ax=ax1)
plt.show()
plt.figure(figsize=(10,8))
plt.title('Regplot showing PLR for MLR Model-lm4')
plt.xlabel('Prediction')
plt.grid(color='b')

# Let's flatten mlr_result and target to 1-D to fit into the Regplot.
yp = mlr_result.squeeze()
tg = target.squeeze()

sns.regplot(yp, tg, color = 'r')
plt.show()<jupyter_output><empty_output><jupyter_text>Prediction and Decision Making  How can we deduce if our Model is correct?   To determine final best fit, we do the following,     1. Determine that our Model results actually make sense.      2. Use Visualization      3. Use Numerical measures for evaluation      4. Compare Models<jupyter_code># For example using the Simple Linear Regression Model, lets view a specific result

x = [30]
x = pd.DataFrame(x)
x
# Now let's pass x as a predictor variable to our SLR Model lm.

lm.predict(x)
# Let's check the coef or slope again of our Model lm
lm.coef_<jupyter_output><empty_output><jupyter_text>The result 13771.30 above seems a fair result. For example the value is not negative, extremely high or extremely low.  If we recall the expression for SLR from above $y = b_0 + b_1x$  _This means that Price ($y^$) = 38423.31 - 821.73 * highway-mpg._  The coefficient or slope ($-821.73$) corresponds to the multiple of the highway-mpg feature.  **As such, an increase in 1(one) unit of highway-mpg would result to a decrease in car price by approximately $821.73.** Pipelines:Data Pipelines simplify the steps of processing the data. We use the module Pipeline to create a pipeline. We also use StandardScaler as a step in our pipeline.<jupyter_code>from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler<jupyter_output><empty_output><jupyter_text>We create the pipeline, by creating a list of tuples including the name of the model or estimator and its corresponding constructor.<jupyter_code>input = [('scale',StandardScaler()),('Poly',PolynomialFeatures(include_bias=False,degree=4)),('model',LinearRegression())]<jupyter_output><empty_output><jupyter_text>we input the list as an argument to the pipeline constructor<jupyter_code>pipe = Pipeline(input)
pipe<jupyter_output><empty_output><jupyter_text>We can normalize the data, perform a transform and fit the model simultaneously.<jupyter_code>pipe.fit(predictors,target)<jupyter_output>/usr/local/lib/python3.6/dist-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.
  return self.partial_fit(X, y)
/usr/local/lib/python3.6/dist-packages/sklearn/base.py:467: DataConversionWarning: Data with input dtype int64 were all converted to float64 by StandardScaler.
  return self.fit(X, y, **fit_params).transform(X)<jupyter_text>Similarly, we can normalize the data, perform a transform and produce a prediction simultaneously<jupyter_code>ypipe = pipe.predict(predictors)
ypipe[:5]
plt.figure(figsize=(10,8))
plt.title('Distplot showing Pipe-Model fit for MLR')
ax1 = sns.distplot(target, hist=False, label='Target', color='r')
sns.distplot(ypipe, hist=False, label='Fitted', color='b', ax=ax1)

plt.figure(figsize=(10,8))
plt.title('Regplot showing Pipe-Model with MLR')
plt.xlabel('Prediction')
plt.grid(color='b')

# Let's flatten ypipe and target to 1-D to fit into the Regplot.
yp = ypipe.squeeze()
tg = target.squeeze()

sns.regplot(yp, tg, color = 'r')
plt.show()<jupyter_output><empty_output><jupyter_text>Measures for In-Sample Model Evaluation  These measures are a way to numerically measure how good the model fits on our data set.  Two important measures to determine the fit of a Model are:  Mean Squared Error (MSE):      To find the MSE, we calculate the differnce between $y$ and $y^$ for all data points in our model prediction. Then we square each difference and sum all the differences and divide that sum by the number of data points or samples predicted.  R-Squared $(R^2)$:      This is also called the coefficient of determination. It's a measure to determine how close the original data is to the fitted regression line. Think about it as comparing a regression model to a simple model of the mean of the data points.      If the variable(s) is a good predictor, then our model should perform better than just the mean. For the most part, it returns values between 0 and 1.      If $R^2$ is negative, it is a possible indication of over-fitting the model.      The closer to 1, the better the model fit and Vice-versa.      $R^2$ = $(1 - \frac{MSE-of-Regression-Line}{MSE-of-The-Average-of-The-Data})$  RESIDUAL VARIANCE AND R-SQUARER-Square, also known as the Coefficient of determination is a commonly used statistic to evaluate model fit. R-square is 1 minus the ratio of residual variability. When the variability of the residual values around the regression line relative to the overall variability is small, the predictions from the regression equation are good. For example, if there is no relationship between the X and Y variables, then the ratio of the residual variability of the Y variable to the original variance is equal to 1.0. Then R-square would be 0. If X and Y are perfectly related then there is no residual variance and the ratio of variance would be 0.0, making R-square = 1. In most cases, the ratio and R-square will fall somewhere between these extremes, that is, between 0.0 and 1.0. This ratio value is immediately interpretable in the following manner. If we have an R-square of 0.4 then we know that the variability of the Y values around the regression line is 1-0.4 times the original variance; in other words we have explained 40% of the original variability, and are left with 60% residual variability. Ideally, we would like to explain most if not all of the original variability. The R-square value is an indicator of how well the model fits the data (e.g., an R-square close to 1.0 indicates that we have accounted for almost all of the variability with the variables specified in the model).[link](http://www.statsoft.com/Textbook/Multiple-Regression) Predicted Values and Residuals	A predicted value is calculated as $^yi=b0+b1xi,1+b2xi,2+…$, where the b values come from statistical software and the x-values are specified by us.	A residual (error) term is calculated as $ei=yi−yi_{hat}$, the difference between an actual and a predicted value of y.	A plot of residuals versus predicted values ideally should resemble a horizontal random band. Departures from this form indicates difficulties with the model and/or data.	Other residual analyses can be done exactly as we did in simple regression. For instance, we might wish to examine a normal probability plot (NPP) of the residuals. Additional plots to consider are plots of residuals versus each x-variable separately. This might help us identify sources of curvature or nonconstant variance.[link](https://newonlinecourses.science.psu.edu/stat501/node/311/) To find the MSE in Python...<jupyter_code># first import mean_squared_error from sklearn

from sklearn.metrics import mean_squared_error
# Lets apply the MSE to the prediction for SLR

error_lm = mean_squared_error(target, Yhat)
error_lm
# Lets apply the MSE to the prediction for SLR using PLR

error_lm3 = mean_squared_error(Y, result)
error_lm3
# Lets apply the MSE to the prediction for MLR 

error_dm = mean_squared_error(target, y_hat)
error_dm
# Let's apply the MSE to the prediction for MLR using PLR

error_lm4 = mean_squared_error(target, mlr_result)
error_lm4
pipe_error = mean_squared_error(target, ypipe)
pipe_error<jupyter_output><empty_output><jupyter_text>To find $R^2$ in Python...      We use the .score() method on our model to find the R squared. Then pass to it our variable or variables and the actual target<jupyter_code># To calculate coefficient of determination for SLR model

lm.score(X,Y)
# To calculate coefficient of determination for SLR model using PLR

lm3.score(X_poly,Y)
# To calculate the coeefficient of determination for MLR model

dm.score(predictors, target)
# to calculate the coefficient of drtermination for MLR model using PLR

lm4.score(predict_fit, target)
pipe.score(predictors,target)<jupyter_output><empty_output><jupyter_text>We can clearly see that so far, The Model with the least MSE and Most $R^2$ is model lm4. No doubt, the best model at the moment.<jupyter_code>test = np.empty([5,3])
chart = pd.DataFrame(test, columns=['Model-Name','MSE', "R-Squ'd"])
chart['Model-Name'] = ['lm','lm3','dm','lm4','pipe']
chart['MSE'] = [31635042.944639895,19986521.832416017,11971078.721508589,2775223.6246514544,2505507.199076076]
chart["R-Squ'd"] = [0.4965911884339175,0.6819542423064463,0.8095040830858192,0.955837833722536,0.9601298343844714]
chart['MSE'] = np.round(chart['MSE'].astype('float64'),2)
chart["R-Squ'd"] = np.round(chart["R-Squ'd"].astype('float64'),4)
chart.sort_values('MSE',ascending=True,inplace=True)
chart.sort_values("R-Squ'd",ascending=False,inplace=True)
chart.index=[1, 2, 3, 4, 5]
chart['Model-Type'] = ['Pipe Model','MLR + PLR','MLR','SLR + PLR', 'SLR']<jupyter_output><empty_output><jupyter_text>See a Summary Table of the Top-Performing Linear Regression Models we created<jupyter_code>chart<jupyter_output><empty_output><jupyter_text>Well after increasing the PolyFeat degree in Pipe-Model to 4, it surpasses the MLR-PLR model which is also at 4 degrees. And performs better in both MSE and $R^2$... Although the difference is very small... Thus The Pipe model is the current best Summary:      Comparing SLR and MLR, a lower MSE does not always imply a better fit in every situation.        MSE for an MLR model will be smaller than MSE for SLR model since the errors of the data would decrease as more variables are included in the model. Also MSE for a Polynomial Regression Model would be smaller than a Linear Regression Model.    A similar inverse relationship holds for $R^2$ too.    Comparing these five models, we conclude that the Pipe model is the best model to be able to predict price from our dataset. This result makes sense, since we have 27 variables in total, and we know that more than one of those variables are potential predictors of the final car price.The second best is The MLR+PLR Model Model Exercises Question 1<jupyter_code># create a Linear regression object

lm1 = LinearRegression()
# Train the model using engine size as independent variable and price as dependent variable

xx = adjusted_df[['engine-size']]
yy = adjusted_df['price']

lm1.fit(xx,yy)
# find the slope and intercept of the model

slope = lm1.coef_
slope
# intercept

intcpt = lm1.intercept_
intcpt<jupyter_output><empty_output><jupyter_text>**What is the equation of the predicted line?**      $y^$ $= b_0 + b_1x_1$ **yhat = -7963.34 + 166.86 * engine-size** Question 2<jupyter_code># create and train a MLR model where the response variable is Price and the predictor variables are normalized losses and 
# Highway-mpg

lm2 = LinearRegression()
adjusted_df['normalized-losses'].unique()
x = adjusted_df['normalized-losses'] == '?'
y = adjusted_df[x]
y

# This shows there are 37 missing values in normalised-losses column alone, that's quite some.
# let's convert them to zero and convert the column to int64

adjusted_df['normalized-losses'].replace('?','0',inplace=True)

# let's finally convert the column normalized losses from object to int
adjusted_df['normalized-losses'] = adjusted_df['normalized-losses'].astype('int64')
# lets set the predictors and response variables.

ex = adjusted_df[['normalized-losses','highway-mpg']]
wy = adjusted_df['price']

# train the model

lm2.fit(ex,wy)
# intercept
lm2.intercept_
# slope
lm2.coef_
adjusted_df.to_csv('adjusted_df.csv')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09498571772722579, 'fraction_numerical': 0.022060088790997006, 'mean_word_length': 4.953288260602336, 'pattern_counts': {'":': 0, '<': 141, '<?xml version=': 0, '>': 141, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '35', 'dir': 'jupyter-structured-clean-dedup', 'id': '635653', 'license': 'BSD-4-Clause-UC', 'n_tokens_mistral': 9457, 'n_tokens_neox': 8804, 'n_words': 3430, 'path': 'model_development_(1).ipynb', 'repo_name': 'Lawrence-Krukrubo/EDA', 'size': '29057'}