<jupyter_start><jupyter_text>Day 2 - Practical ✍ Advanced ML Workshop--- Resale Housing DatasetWelcome to the second practical session of SPAI's Advanced Machine Learning Workshop. In this practical, you will experience evaluating the performance the model through cross validation and gain better intuition of bias and variance problem faced by a model. The goal of this dataset is to predict the resale housing price based on the housing features.  ⚙ *Basic Initialization*Please do not make any changes to this section.   This section is required for the demo to work and not running it would result in errors 🚨.   However, if you would like to understand the code, feel free to do so 😊  Import Libraries<jupyter_code># Importing necessary libraries
import numpy as np
import pandas as pd

# Sklearn Functions
# Data Partition
from sklearn.model_selection import train_test_split
# Preprocessing
from sklearn.preprocessing import OneHotEncoder
# Pipeline
from imblearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer<jupyter_output><empty_output><jupyter_text>> ⚠ Please first ensure that yellowbrick has been installed. If not, run the following command:   ```bash!pip install yellowbrick --upgrade```<jupyter_code>!pip install yellowbrick --upgrade<jupyter_output>Requirement already satisfied: yellowbrick in /usr/local/lib/python3.7/dist-packages (1.3.post1)
Requirement already satisfied: matplotlib!=3.0.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from yellowbrick) (3.2.2)
Requirement already satisfied: numpy<1.20,>=1.16.0 in /usr/local/lib/python3.7/dist-packages (from yellowbrick) (1.19.5)
Requirement already satisfied: scipy>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from yellowbrick) (1.4.1)
Requirement already satisfied: scikit-learn>=0.20 in /usr/local/lib/python3.7/dist-packages (from yellowbrick) (0.22.2.post1)
Requirement already satisfied: cycler>=0.10.0 in /usr/local/lib/python3.7/dist-packages (from yellowbrick) (0.10.0)
Requirement already satisfied: six in /usr/local/lib/python3.7/dist-packages (from cycler>=0.10.0->yellowbrick) (1.15.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib!=3.0.0,>=2.0.2->yellowbrick) (1.3.1)
Requirement already satisfied: pypa[...]<jupyter_text>Import Data<jupyter_code># Init data 🏠
house_df = pd.read_csv('https://raw.githubusercontent.com/SPAI-Team/Advanced-ML-Workshop-2021/main/Datasets/HousingDataset.csv')
house_df = house_df.drop(columns=['Flat Model', 'Block', 'Full Address', 'Lease Commencement Date', 'Remaining Lease Duration', 'Street Name', 'Month of Resale'])<jupyter_output><empty_output><jupyter_text>Data Preprocessing<jupyter_code># Removes null data as there are very few rows of null data
house_df = house_df.dropna()<jupyter_output><empty_output><jupyter_text>Data Splitting<jupyter_code>X, y = house_df.drop(columns=['Resale Price']), house_df['Resale Price']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state = 24)<jupyter_output><empty_output><jupyter_text>Modelling UtilitiesThe following function returns a `Pipeline` instance that contains the `ColumnTransformer` for preprocessing steps and a customizable estimator.Creating a function which returns a pipeline with the entire preprocessing steps could be helpful for you to reduce several lines of code.Example:``` python from sklearn.linear_model import LogisticRegression  Import logistic regressionlogex_pipe = custom_pipeline(LogisticRegression())  Define the model with custom pipelinelogex_pipe.fit(X_train, y_train)  Train model```<jupyter_code># Use of function allows us to parse ANY model through the pipeline
def custom_pipeline(model):
    return Pipeline(steps=[
        ('encoder', ColumnTransformer([
                    ('onehot', OneHotEncoder(drop='first', sparse=False), ['Flat Type', 'Storey Range', 'Town'])
                ], remainder='passthrough')
        ), 
        ('model', model)
    ])<jupyter_output><empty_output><jupyter_text>---  Exercise 1️⃣ Cross ValidationIn this exercise, you will learn how to apply cross validation, and get the results out of it.  Task1. Import the `cross_validate` library from model_selection2. Apply cross validation with the `cross_validate` function you just imported and save the result in the variable `lr_results`. In the `cross_validate` function, parse the following parameters:    - The Linear Regression model (the Linear Regression model has already been defined as the variable `lr`)    - `X_train` and `y_train`    - `scoring` set to `scoring_metrics` list (already defined)    - `n_jobs` set to `-1`<jupyter_code>from sklearn.linear_model import LinearRegression
lr = custom_pipeline(LinearRegression())

scoring_metrics = ['neg_root_mean_squared_error', 'r2']


### Write your code here ###
# import the cross_validate library from model_selection in sklearn library
from sklearn.______ import ______

# Apply cross validation with k=5 using the 'lr' model, scoring set to 'scoring_metrics' variable 
# Provided and initialize it to lr_results.
lr_results = ______(______, ______, ______, scoring=______, n_jobs=______)

### End ###

# Prints result
print(lr_results)<jupyter_output>{'fit_time': array([0.99649477, 1.00326538, 0.92081738, 0.87209392, 0.49279404]), 'score_time': array([0.07644844, 0.06274652, 0.04521728, 0.04672551, 0.02959824]), 'test_neg_root_mean_squared_error': array([-65658.70121082, -65131.04246448, -65350.2868511 , -65359.72012704,
       -65823.72153353]), 'test_r2': array([0.7781292 , 0.78078019, 0.77516699, 0.7791109 , 0.77635292])}<jupyter_text>Task1. Find the average RMSE and save it as variable `lr_rmse`.     - Use the key `test_neg_root_mean_squared_error` to extract the array from `lr_results`    - Use `np.mean()` to find the average of a list2. Find the average r2 and save it as variable `lr_r2`.    - Use the key `test_r2` to extract the array from `lr_results`    - Use `np.mean()` to find the average of a list<jupyter_code>'''
HINT:
lr_results is a Python Dictionary

sample_dictionary = {
    'key1':'value1',
    'key2':'value2'
}

To access the value in a dictionary you need to call it by its key:
sample_dictionary['key1'] => 'value1'
'''

### Write your code here ###
# Find out the average of the root mean squared error (RMSE) and r2 score from lr_results
lr_rmse = np.______(______['______'])
lr_r2 = np.______(______['______'])

### End ###

# Prints result
print('Model for Decision Tree')
print('RMSE is', lr_rmse)
print('R2 score is', lr_r2)<jupyter_output>Model for Decision Tree
RMSE is -65464.694437393264
R2 score is 0.7779080411920675<jupyter_text>TaskBased off the previous two cells, try applying cross validation using the Decision Tree model instead. Feel free to refer to the steps in the previous cells as the solution are similar.<jupyter_code>'''
Hint:
1. Apply cross validation with decision tree regressor [Stored in dtr] and save the result in dtr_results
2. Find the average RMSE and save the result in dtr_rmse
3. Find the average r2 and save the result in dtr_r2
'''
from sklearn.tree import DecisionTreeRegressor
dtr = custom_pipeline(DecisionTreeRegressor())


### Write your code here ###
# Apply cross validate with k=5 using the 'dtr' model this time, scoring set to 'scoring_metrics' variable
# Provided and initialize it to dtr_results.
dtr_results =  ______(______, ______, ______, scoring=______, n_jobs=______)

# Print out the average of the root mean sqaured error (RMSE) and r2 score from dtr_results
dtr_rmse = np.______(______['______'])
dtr_r2 = np.______(______['______'])
### End ###

# Prints result
print('Model for Decision Tree')
print('RMSE is', dtr_rmse)
print('R2 score is', dtr_r2)<jupyter_output>Model for Decision Tree
RMSE is -49601.1397875726
R2 score is 0.8725001531072039<jupyter_text>---  Exercise 2️⃣ Model Learning CurveIn this exercise you will learn how to plot a model's learning curve.  Task1. Import the `LearningCurve` function under `model_selection` from `yellowbrick`2. Set the variable `viz` to the `LearningCurve` functin you just imported and parse the following    - the model `dtr`    - `cv=5` and `scoring=neg_root_mean_squared_error`    - `n_jobs=-1` for faster training times3. Apply the `.fit` method on the `viz` variable, and parse `X_train` and `y_train` in the `.fit` method4. Answer the following:     - Is this model overfitting, underfitting or it has a good fit<jupyter_code>from sklearn.tree import DecisionTreeRegressor
dtr = DecisionTreeRegressor()
X_new = X.drop(columns=['Flat Type', 'Storey Range', 'Town'])
X_train, X_test, y_train, y_test = train_test_split(X_new, y, test_size=0.2)

### Write your code here ###
# from the yellowbrick library under model_selection, import LearningCurve
from yellowbrick.______ import ______

# initialize LearningCurve to the variable viz
# Parse in the 'dtr' model, with 5 folds cross validation, scoring of 'neg_root_mean_squared_error'
# 💡 Tip: Set n_jobs=-1 for faster results
viz = ______(______, cv=______, scoring=______, n_jobs=______)

# Fit viz to the target and feature variables
viz.fit(______, ______)

# Display viz
viz.show()

### End ###


# 🙋‍♂️🙋‍♀️ Follow Up Question:
# > Is this model Overfitting, Underfitting or Good fit
# Answer: _________<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10145703878294407, 'fraction_numerical': 0.042639811442039856, 'mean_word_length': 5.31596752368065, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 50, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '173823', 'license': 'MIT', 'n_tokens_mistral': 3231, 'n_tokens_neox': 2969, 'n_words': 993, 'path': 'Advance Machine Learning Workshop/Day 2_Bias and Variance/Practical Notebooks/Day 2 Practical Notebook (Participant).ipynb', 'repo_name': 'TYH71/Advance_ML_Workshop', 'size': '9334'}