<jupyter_start><jupyter_text>Model buildingWe will finally build our machine learnig model, utilising the engineered data and the pre-selected features.  Imports<jupyter_code># to handle datasets
import pandas as pd 
import numpy as np 

# for plotting 
import matplotlib.pyplot as plt 

# to build the model 
from sklearn.linear_model import Lasso 

# to evaluate the model 
from sklearn.metrics import mean_squared_error, r2_score 
from math import sqrt 

# to visualize all the columns in the dataframe
pd.pandas.set_option('display.max_columns', None)<jupyter_output><empty_output><jupyter_text>Gathering data<jupyter_code>X_train = pd.read_csv('xtrain.csv')
X_test = pd.read_csv('xtest.csv')

X_train.head()
X_train['LotFrontage']
# capture the target (remember that is log transformed)
y_train = X_train['SalePrice']
y_test = X_test['SalePrice']<jupyter_output><empty_output><jupyter_text>Load pre-selected features<jupyter_code>features = pd.read_csv('selected_features.csv')
features = features['0'].to_list()

# We will add one additional feature to the ones we selected in the prvious notebook: LotFrontage
# because it needs key feature engineering steps that we want to
# discuss further during the deployment part of the course. 

features = features + ['LotFrontage']
features
X_train = X_train[features]
X_test = X_test[features]<jupyter_output><empty_output><jupyter_text>Regularised linear regression: Lasso<jupyter_code>lin_model = Lasso(alpha=0.005, random_state=0)

# train the model 
lin_model.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Evaluate the model<jupyter_code># Remember that we log trasnformed the output (SalePrice)

# In order to get the true performance of the Lasso
# we need to transform both the target and hte predictions

# We will evaluate performance using the mean squared error and the root of the mean squared error and r2

# make predictiosn for train set

pred = lin_model.predict(X_train)

# Determine ms and rmse

print('train mse: {}'.format(
    int(mean_squared_error(np.exp(y_train), np.exp(pred)))
    )
)

print('train rmse: {}'.format(int(
    sqrt(mean_squared_error(np.exp(y_train), np.exp(pred))))
    )
)

print('train r2: {}'.format(
    r2_score(np.exp(y_train), np.exp(pred))
    )
)


# make predictions for test set

pred = lin_model.predict(X_test)

# determine mse and rmse

print('test mse: {}'.format(
    int(mean_squared_error(np.exp(y_test), np.exp(pred)))
    )
)

print('test rmse: {}'.format(
    int(sqrt(mean_squared_error(np.exp(y_test), np.exp(pred))))
    )
)

print('test r2: {}'.format(
    r2_score(np.exp(y_test), np.exp(pred))
    )
)
print('Average house price: ', int(np.exp(y_train).median()))
# let's evaluate our predictions respect to the real sale price
plt.scatter(y_test, lin_model.predict(X_test))
plt.xlabel('True House Price')
plt.ylabel('Predicted House Price')
plt.title('Evaluation of Lasso Predictions')
# let's evaluate the distrubtion of the errors: 
# they should be fairly normally distributed

errors = y_test - lin_model.predict(X_test)
errors.hist(bins=30)
# The distribution of the errors follows quite closely a gaussian distribution. That suggest that our model is doing a good job as well.<jupyter_output><empty_output><jupyter_text>Feature importanceFinally, just for fun, let's look at the feature importance<jupyter_code>importance = pd.Series(np.abs(lin_model.coef_.ravel()))
importance.index = features 
importance.sort_values(inplace=True, ascending=False)
importance.plot.bar(figsize=(18,6))
plt.ylabel('Lasso Coefficients')
plt.title('Feature Importance')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10700496414782129, 'fraction_numerical': 0.004688361831218974, 'mean_word_length': 5.869318181818182, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '368738', 'license': 'MIT', 'n_tokens_mistral': 1245, 'n_tokens_neox': 1192, 'n_words': 330, 'path': 'section2/model_building.ipynb', 'repo_name': 'Jeffresh/deployment-of-machine-learning-models', 'size': '3626'}