<jupyter_start><jupyter_text>_Lambda School Data Science_ Regression Sprint Challenge For this Sprint Challenge, you'll predict the price of used cars. The dataset is real-world. It was collected from advertisements of cars for sale in the Ukraine in 2016.The following import statements have been provided for you, and should be sufficient. But you may not need to use every import. And you are permitted to make additional imports.<jupyter_code>%matplotlib inline
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import seaborn as sns
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeRegressor
import statsmodels.api as sm
from statsmodels.stats.outliers_influence import variance_inflation_factor<jupyter_output><empty_output><jupyter_text>[The dataset](https://raw.githubusercontent.com/ryanleeallred/datasets/master/car_regression.csv) contains 8,495 rows and 9 variables:- make: manufacturer brand- price: seller’s price in advertisement (in USD)- body: car body type- mileage: as mentioned in advertisement (‘000 Km)- engV: rounded engine volume (‘000 cubic cm)- engType: type of fuel- registration: whether car registered in Ukraine or not- year: year of production- drive: drive typeRun this cell to read the data:<jupyter_code>df = pd.read_csv('https://raw.githubusercontent.com/ryanleeallred/datasets/master/car_regression.csv')
print(df.shape)
df.sample(10)<jupyter_output>(8495, 9)<jupyter_text>Predictive Modeling with Linear Regression  1.1 Split the data into an X matrix and y vector (`price` is the target we want to predict).<jupyter_code>target = 'price'
features = df.columns.drop('price')

y = df[target]
X = df[features]<jupyter_output><empty_output><jupyter_text>1.2 Split the data into test and train sets, using `train_test_split`.You may use a train size of 80% and a test size of 20%.<jupyter_code>X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=.80, test_size=.20, random_state=3)<jupyter_output><empty_output><jupyter_text>1.3 Use scikit-learn to fit a multiple regression model, using your training data.Use `year` and one or more features of your choice. You will not be evaluated on which features you choose. You may choose to use all features.<jupyter_code>features = ['year', 'mileage']

model = LinearRegression()
model.fit(X_train[features], y_train)<jupyter_output><empty_output><jupyter_text>1.4 Report the Intercept and Coefficients for the fitted model.<jupyter_code>print('intercept: ', model.intercept_, 'coefficients: ',  model.coef_)<jupyter_output>intercept:  -2167340.7899214127 coefficients:  [1091.44589818  -45.43216792]<jupyter_text>1.5 Use the test data to make predictions.<jupyter_code>y_pred = model.predict(X_test[features])
y_pred<jupyter_output><empty_output><jupyter_text>1.6 Use the test data to get both the Root Mean Square Error and $R^2$ for the model. You will not be evaluated on how high or low your scores are.<jupyter_code>rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2 = r2_score(y_test, y_pred)

print('rmse: ', rmse, 'r2: ', r2)<jupyter_output>rmse:  20204.382033228845 r2:  0.19872478227172274<jupyter_text>1.7 How should we interpret the coefficient corresponding to the `year` feature?One sentence can be sufficient For each year that the car is newer (more recent), the price goes up the coefficient amount.  In this case, $1091.44  1.8 How should we interpret the Root Mean Square Error?One sentence can be sufficient The rmse explains how close the predicted data is to the actual test data that was thrown to it.  It indicates how well the model fits the data.  In this case, rmse of 20204.38.. can be explaiend by saying that the predictions will be off by that much on average. (not great)  1.9 How should we interpret the $R^2$?One sentence can be sufficient R2 is another measure of fit. An R2 of .1987.. can be interpreted by saying that this model improves your chances of predicting the price by 19%, compared to picking at random.  Log-Linear and Polynomial Regression  2.1 Engineer a new variable by taking the log of the price varible.<jupyter_code>df['ln_price'] = np.log(df['price'])<jupyter_output><empty_output><jupyter_text>2.2 Visualize scatterplots of the relationship between each feature versus the log of price, to look for non-linearly distributed features.You may use any plotting tools and techniques.<jupyter_code>target = 'ln_price'
features = df.columns.drop(['price', 'ln_price'])

for feature in features:
  sns.scatterplot(x=feature, y=target, data=df, alpha=.2)
  plt.show()<jupyter_output><empty_output><jupyter_text>2.3 Create polynomial feature(s)You will not be evaluated on which feature(s) you choose. But try to choose appropriate features.<jupyter_code>df['year_squared'] = df['year']**2
df['engV_squared'] =df['engV']**2
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=.80, test_size=.20, random_state=3)<jupyter_output><empty_output><jupyter_text>2.4 Use the new log-transformed y variable and your x variables (including any new polynomial features) to fit a new linear regression model. Then report the: intercept, coefficients, RMSE, and $R^2$.<jupyter_code>features = df.columns.drop(['price', 'ln_price'])

y = df[target]
X = df[features]

model = LinearRegression()
model.fit(X_train[features], y_train)

y_pred = model.predict(X_test[features])

rmse = np.sqrt(mean_squared_error(y_test, y_pred))
r2 = r2_score(y_test, y_pred)

print('rmse: ', rmse, 'r2: ', r2)
print('intercept: ', model.intercept_, 'coefficients: ', model.coef_)<jupyter_output>rmse:  0.46841566213373276 r2:  0.7678745574274062
intercept:  6715.210670779128 coefficients:  [-1.28551673e-03 -7.64384160e-02  5.25769035e-04  2.20486046e-01
 -4.72423776e-02  6.69635998e-01 -6.79946651e+00  2.64608934e-01
  1.72278243e-03 -2.23411484e-03]<jupyter_text>2.5 How do we interpret coefficients in Log-Linear Regression (differently than Ordinary Least Squares Regression)?One sentence can be sufficient Instead of the change by the coefficient being explained in units, the change is explained in percentages.   Decision Trees  3.1 Use scikit-learn to fit a decision tree regression model, using your training data.Use one or more features of your choice. You will not be evaluated on which features you choose. You may choose to use all features.You may use the log-transformed target or the original un-transformed target. You will not be evaluated on which you choose.<jupyter_code>target = 'ln_price'
features = ['year_squared', 'engV_squared']

y = df[target]
X = df[features]
tree = DecisionTreeRegressor(max_depth=3)
tree.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>3.2 Use the test data to get the $R^2$ for the model. You will not be evaluated on how high or low your scores are.<jupyter_code>print('R2: ', tree.score(X_test, y_test))<jupyter_output>R2:  0.6896980383957804<jupyter_text>Regression Diagnostics  4.1 Use statsmodels to run a log-linear or log-polynomial linear regression with robust standard errors.<jupyter_code>target = 'ln_price'
features = df.columns.drop(['price', 'ln_price'])

y = df[target]
X = df[features]

model = sm.OLS(y, sm.add_constant(X))
results = model.fit()
print(results.summary())<jupyter_output>OLS Regression Results                            
==============================================================================
Dep. Variable:               ln_price   R-squared:                       0.744
Model:                            OLS   Adj. R-squared:                  0.743
Method:                 Least Squares   F-statistic:                     2462.
Date:                Fri, 03 May 2019   Prob (F-statistic):               0.00
Time:                        16:08:32   Log-Likelihood:                -5940.3
No. Observations:                8495   AIC:                         1.190e+04
Df Residuals:                    8484   BIC:                         1.198e+04
Df Model:                          10                                         
Covariance Type:            nonrobust                                         
================================================================================
                   coef    std err          t     [...]<jupyter_text>4.2 Calculate the Variance Inflation Factor (VIF) of our X variables.  Do we have multicollinearity problems?One sentence can be sufficient<jupyter_code>vif = [variance_inflation_factor(sm.add_constant(X).values, i) for i in range(len(sm.add_constant(X).columns))]
pd.Series(vif, sm.add_constant(X).columns)<jupyter_output>/usr/local/lib/python3.6/dist-packages/numpy/core/fromnumeric.py:2389: FutureWarning: Method .ptp is deprecated and will be removed in a future version. Use numpy.ptp instead.
  return ptp(axis=axis, out=out, **kwargs)
{'fraction_non_alphanumeric': 0.10613207547169812, 'fraction_numerical': 0.04784366576819407, 'mean_word_length': 4.28486646884273, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 58, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '605410', 'license': 'MIT', 'n_tokens_mistral': 3010, 'n_tokens_neox': 2709, 'n_words': 907, 'path': 'Regression Sprint Challenge.ipynb', 'repo_name': 'NicoMontoya/DS-Unit-2-Sprint-2-Regression', 'size': '8904'}