<jupyter_start><jupyter_text>Linear Regression Project Exercise - Solutions **TASK: Run the cells under the Imports and Data section to make sure you have imported the correct general libraries as well as the correct datasets.     Later on you may need to run further imports from scikit-learn.**  Imports<jupyter_code>import pandas as pd       
import numpy as np 
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score
from sklearn.model_selection import train_test_split, GridSearchCV
import warnings
warnings.filterwarnings('ignore')
pd.set_option('display.max_columns', 500)<jupyter_output><empty_output><jupyter_text>Data<jupyter_code>df = pd.read_csv("AMES_Final_DF.csv")
df.head()
df.shape
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 2925 entries, 0 to 2924
Columns: 274 entries, Lot Frontage to Sale Condition_Partial
dtypes: float64(11), int64(263)
memory usage: 6.1 MB<jupyter_text>**TASK: The label we are trying to predict is the SalePrice column. Separate out the data into X features and y labels**<jupyter_code>X = df.drop('SalePrice',axis=1)
y = df['SalePrice']<jupyter_output><empty_output><jupyter_text>**TASK: Use scikit-learn to split up X and y into a training set and test set. Since we will later be using a Grid Search strategy, set your test proportion to 10%. To get the same data split as the solutions notebook, you can specify random_state = 101**<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.20, random_state=101)<jupyter_output><empty_output><jupyter_text>**TASK: The dataset features has a variety of scales and units. For optimal regression performance, scale the X features. Take carefuly note of what to use for .fit() vs what to use for .transform()**<jupyter_code>from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
scaled_X_train = scaler.fit_transform(X_train)
scaled_X_test = scaler.transform(X_test)
autoscout=pd.DataFrame(scaled_X_train, columns=X_train.columns)
autoscout.head()
from sklearn.linear_model import Lasso
from sklearn.linear_model import LassoCV
def eval_metrics(actual, pred):
    rmse = np.sqrt(mean_squared_error(actual, pred))
    mae = mean_absolute_error(actual, pred)
    mse = mean_squared_error(actual, pred)
    score = r2_score(actual, pred)
    return print("r2_score:", score, "\n","mae:", mae, "\n","mse:",mse, "\n","rmse:",rmse)
lasso_model = Lasso()
lasso_model.fit(scaled_X_train, y_train)
y_pred = lasso_model.predict(scaled_X_test)
eval_metrics(y_test, y_pred)
from sklearn.model_selection import cross_val_score
cv_scores = cross_val_score(estimator=lasso_model, X=scaled_X_train, y=y_train, cv=10)
cv_scores.mean()
cv_scores = cross_val_score(estimator=lasso_model, X=scaled_X_train, y=y_train, scoring = "neg_mean_squared_error", cv=10)
-cv_scores.mean()
np.sqrt(-cv_scores.mean())
y.mean()
alpha_space = np.linspace(0.1, 20, 50)
lasso_cv_model = LassoCV(alphas = alpha_space, cv = 5).fit(scaled_X_train, y_train)
lasso_cv_model.alpha_
alpha_space = range(1,500)
lasso_cv_model = LassoCV(alphas = alpha_space, cv = 5).fit(scaled_X_train, y_train)
lasso_cv_model.alpha_
from sklearn.linear_model import LassoCV
from yellowbrick.regressor import AlphaSelection

# Create a list of alphas to cross-validate against
alpha_space = range(1,500)

# Instantiate the linear model and visualizer
model = LassoCV(alphas=alpha_space)
visualizer = AlphaSelection(model)
visualizer.fit(scaled_X_train, y_train)
visualizer.show()
lasso_model = Lasso(213).fit(scaled_X_train, y_train)
y_pred = lasso_model.predict(scaled_X_test)
eval_metrics(y_test, y_pred)
lasso_model = Lasso(500).fit(scaled_X_train, y_train)
y_pred = lasso_model.predict(scaled_X_test)
eval_metrics(y_test, y_pred)
from yellowbrick.features import RadViz
from yellowbrick.model_selection import FeatureImportances

# Load the regression dataset

# Title case the feature for better display and create the visualizer
labels = list(map(lambda s: s.title(), X.columns))
viz = FeatureImportances(lasso_model, labels=labels, relative=False)
visualizer = RadViz(size=(1080, 7200))

# Fit and show the feature importances
viz.fit(scaled_X_train, y_train)
viz.show()
# Create features DataFrame
feature_coef = pd.DataFrame({'features' : X.columns, 'coef' : lasso_model.coef_})
significant_features = feature_coef.loc[feature_coef.coef != 0, :]
len(significant_features)
significant_features.sort_values(by="coef", ascending=False)
significant_features["features"]
X_new=X[significant_features.features]
X_new.shape
y.shape
X_new.head()<jupyter_output><empty_output><jupyter_text>**Linear Regression with reduced columns**<jupyter_code>from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X_new, y, test_size = 0.2, random_state = 0)
from sklearn.linear_model import LinearRegression
lm = LinearRegression()
lm.fit(X_train,y_train)
y_pred = lm.predict(X_test)
eval_metrics(y_test, y_pred)<jupyter_output>r2_score: 0.927933951626356 
 mae: 14492.719008006614 
 mse: 443505048.5552273 
 rmse: 21059.55955273584<jupyter_text>**Linear Regression with original columns**<jupyter_code>#let's try the linear reg with original dataset.
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 0)
from sklearn.linear_model import LinearRegression
lm = LinearRegression()
lm.fit(X_train,y_train)
y_pred = lm.predict(X_test)
eval_metrics(y_test, y_pred)<jupyter_output>r2_score: 0.9132021061163716 
 mae: 15745.198488637965 
 mse: 534166990.00563675 
 rmse: 23112.052916295357
{'fraction_non_alphanumeric': 0.09579199452617174, 'fraction_numerical': 0.03711939787889155, 'mean_word_length': 7.235211267605634, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 27, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '317861', 'license': 'MIT', 'n_tokens_mistral': 2225, 'n_tokens_neox': 2038, 'n_words': 498, 'path': 'Data Science and Machine Learning/Machine-Learning-In-Python-THOROUGH/MACHINE_LEARNING/05_LINEAR_REGRESSION_LASSO_AMES_FINAL_DATASET.ipynb', 'repo_name': 'okara83/Becoming-a-Data-Scientist', 'size': '5846'}