<jupyter_start><jupyter_text>Lasso Regression algorithm  Lasso regression is a regularization set of rules which may be used to remove irrelevant noises and do characteristic choice and subsequently regularize a model. The word “LASSO” stands for Least Absolute Shrinkage and Selection Operator. It is a statistical formula for the regularisation of data models and feature selection.Lasso Regression uses L1 regularization technique (will be discussed later in this article). It is used when we have more number of features because it automatically performs feature selection. *Lasso regression penalizes less important features of your dataset and makes their respective coefficients zero, thereby eliminating them. Thus it provides you with the benefit of feature selection and simple model creation. So, if the dataset has high dimensionality and high correlation, lasso regression can be used.*  Implementation of Lasso regression  Import Required Libraries<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('ggplot')
import warnings; warnings.simplefilter('ignore')<jupyter_output><empty_output><jupyter_text>Load the Dataset<jupyter_code>df = pd.read_csv("C:/Users/S_The/Downloads/Auto.csv")
df.head()
df = df.iloc[0:200]
df = df.drop(['name'], axis=1)
df.info()
df['origin'] = pd.Categorical(df['origin'])
df['horsepower'] = pd.to_numeric(df['horsepower'], errors='coerce')
print(df.isnull().sum())
df = df.dropna()<jupyter_output><empty_output><jupyter_text>Standardization **It is important to standardize the features by removing the mean and scaling to unit variance.If a feature has a variance that is orders of magnitude larger that others, it might dominate the objective function and make the estimator unable to learn from other features correctly as expected.**<jupyter_code>dfs = df.astype('int')
dfs.info()
dfs.columns
from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()
dfs[['cylinders', 'displacement', 'horsepower',
     'weight', 'acceleration', 'year', 'origin']] = scaler.fit_transform(dfs[['cylinders',
                                                                              'displacement',
                                                                              'horsepower',
                                                                              'weight',
                                                                              'acceleration',
                                                                              'year', 'origin']])
dfs.head(5)
X = dfs.drop(['mpg'], axis=1)
y = dfs['mpg']<jupyter_output><empty_output><jupyter_text>Split data **Split the data set into train and test sets (use X_train, X_test, y_train, y_test), with the first 75% of the data for training and the remaining for testing.**<jupyter_code>from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=10)<jupyter_output><empty_output><jupyter_text>Lasso regression **Apply Lasso regression on the training set with the regularization parameter lambda = 0.5 and print the R^2-score for the training and test set. Comment on your findings.**<jupyter_code>from sklearn.linear_model import Lasso

reg = Lasso(alpha=0.5)
reg.fit(X_train, y_train)
print('Lasso Regression: R^2 score on training set', reg.score(X_train, y_train)*100)
print('Lasso Regression: R^2 score on test set', reg.score(X_test, y_test)*100)<jupyter_output>Lasso Regression: R^2 score on training set 82.49741060950073
Lasso Regression: R^2 score on test set 85.49734440925532<jupyter_text>Lasso with different lambdas *Apply the Lasso regression on the training set with the following λ parameters: (0.001, 0.01, 0.1, 0.5, 1, 2, 10). Evaluate the R^2 score for all the models you obtain on both the train and test sets.*<jupyter_code>lambdas = (0.001, 0.01, 0.1, 0.5, 1, 2, 10)
l_num = 7
pred_num = X.shape[1]

# prepare data for enumerate
coeff_a = np.zeros((l_num, pred_num))
train_r_squared = np.zeros(l_num)
test_r_squared = np.zeros(l_num)
for ind, i in enumerate(lambdas):    
    reg = Lasso(alpha = i)
    reg.fit(X_train, y_train)

    coeff_a[ind,:] = reg.coef_
    train_r_squared[ind] = reg.score(X_train, y_train)
    test_r_squared[ind] = reg.score(X_test, y_test)<jupyter_output><empty_output><jupyter_text>Plot values as a function of lambda<jupyter_code>plt.figure(figsize=(10, 8))
plt.plot(train_r_squared, 'bo-', label=r'$R^2$ Training set', color="darkblue", alpha=0.6, linewidth=3)
plt.plot(test_r_squared, 'bo-', label=r'$R^2$ Test set', color="darkred", alpha=0.6, linewidth=3)
plt.xlabel('Lamda index'); plt.ylabel(r'$R^2$')
plt.xlim(0, 6)
plt.title(r'Evaluate lasso regression with lamdas: 0 = 0.001, 1= 0.01, 2 = 0.1, 3 = 0.5, 4= 1, 5= 2, 6 = 10')
plt.legend(loc='best')
plt.grid()<jupyter_output><empty_output><jupyter_text>Identify best lambda and coefficients *Store your test data results in a DataFrame and indentify the lambda where the R^2 has it’s maximum value in the test data. Fit a Lasso model with this lambda parameter (use the training data) and obtain the corresponding regression coefficients. Furthermore, obtain the mean squared error for the test data of this model.*<jupyter_code>df_lam = pd.DataFrame(test_r_squared*100, columns=['R_squared'])
df_lam['lambda'] = (lambdas)
# returns the index of the row where column has maximum value.
df_lam.loc[df_lam['R_squared'].idxmax()]
# Coefficients of best model
reg_best = Lasso(alpha = 0.1)
reg_best.fit(X_train, y_train)
reg_best.coef_
from sklearn.metrics import mean_squared_error
mean_squared_error(y_test, reg_best.predict(X_test))<jupyter_output><empty_output><jupyter_text>Cross Validation *Evaluate the performance of a Lasso regression for different regularization parameters λ using 5-fold cross validation on the training set (module: from sklearn.model_selection import cross_val_score) and plot the cross-validation (CV) R2scores of the training and test data as a function of λ.*<jupyter_code>l_min = 0.05
l_max = 0.2
l_num = 20
lambdas = np.linspace(l_min,l_max, l_num)

train_r_squared = np.zeros(l_num)
test_r_squared = np.zeros(l_num)

pred_num = X.shape[1]
coeff_a = np.zeros((l_num, pred_num))
from sklearn.model_selection import cross_val_score

for ind, i in enumerate(lambdas):    
    reg = Lasso(alpha = i)
    reg.fit(X_train, y_train)
    results = cross_val_score(reg, X, y, cv=5, scoring="r2")

    train_r_squared[ind] = reg.score(X_train, y_train)    
    test_r_squared[ind] = reg.score(X_test, y_test)
plt.figure(figsize=(10, 8))
plt.plot(train_r_squared, 'bo-', label=r'$R^2$ Training set', color="darkblue", alpha=0.6, linewidth=3)
plt.plot(test_r_squared, 'bo-', label=r'$R^2$ Test set', color="darkred", alpha=0.6, linewidth=3)
plt.xlabel('Lamda value'); plt.ylabel(r'$R^2$')
plt.xlim(0, 19)
plt.title(r'Evaluate 5-fold cv with different lamdas')
plt.legend(loc='best')
plt.grid()<jupyter_output><empty_output><jupyter_text>Best Model *Finally, store your test data results in a DataFrame and identify the lambda where the R^2 has it’s maximum value in the test data. Fit a Lasso model with this lambda parameter (use the training data) and obtain the corresponding regression coefficients. Furthermore, obtain the mean squared error for the test data of this model.*<jupyter_code>df_lam = pd.DataFrame(test_r_squared*100, columns=['R_squared'])
df_lam['lambda'] = (lambdas)
# returns the index of the row where column has maximum value.
df_lam.loc[df_lam['R_squared'].idxmax()]
reg_best = Lasso(alpha = 0.144737)
reg_best.fit(X_train, y_train)
Lasso(alpha=0.144737, copy_X=True, fit_intercept=True, max_iter=1000, normalize=False, positive=False, precompute=False, random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

from sklearn.metrics import mean_squared_error

mean_squared_error(y_test, reg_best.predict(X_test))
reg_best.coef_<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10573073073073073, 'fraction_numerical': 0.024024024024024024, 'mean_word_length': 4.546842470506593, 'pattern_counts': {'":': 0, '<': 40, '<?xml version=': 0, '>': 40, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 16, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '137048', 'license': 'MIT', 'n_tokens_mistral': 2752, 'n_tokens_neox': 2529, 'n_words': 786, 'path': 'Datascience_With_Python/Machine Learning/Videos/Lasso Regression algorithm/Lasso Regression algorithm.ipynb', 'repo_name': 'vishnupriya129/winter-of-contributing', 'size': '7992'}