<jupyter_start><jupyter_text>Task 3  Imports<jupyter_code>import numpy as np
import pandas as pd
from joblib import dump, load
from sklearn.metrics import mean_squared_error
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
import matplotlib.pyplot as plt
plt.style.use('ggplot')
plt.rcParams.update({'font.size': 16, 'axes.labelweight': 'bold', 'figure.figsize': (8,6)})<jupyter_output>Matplotlib created a temporary config/cache directory at /tmp/matplotlib-bptof89q because the default path (/home/jupyter-student54/.cache/matplotlib) is not a writable directory; it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.<jupyter_text>Part 1: Recall as a final goal of this project. We want to build and deploy ensemble machine learning models in the cloud, where features are outputs of different climate models and the target is the actual rainfall observation. In this milestone, you'll actually build these ensemble machine learning models in the cloud.  **Your tasks:**1. Read the data CSV from your s3 bucket. 2. Drop rows with nans. 3. Split the data into train (80%) and test (20%) portions with `random_state=123`. 4. Carry out EDA of your choice on the train split. 5. Train ensemble machine learning model using `RandomForestRegressor` and evaluate with metric of your choice (e.g., `RMSE`) by considering `Observed` as the target column. 6. Discuss your results. Are you getting better results with ensemble models compared to the individual climate models? > Recall that individual columns in the data are predictions of different climate models.   1. Loading csv<jupyter_code>## You could download it from your bucket, or you can use the file that I have in my bucket. 
## You should be able to access it from my bucket using your key and secret
aws_credentials ={"key": "","secret": ""} 
df = pd.read_csv("s3://mds-s3-student42/output/ml_data_SYD.csv", storage_options=aws_credentials)<jupyter_output><empty_output><jupyter_text>2. Drop nan rows<jupyter_code>df = df.dropna(axis='rows')
df = df.drop(columns=['time'])<jupyter_output><empty_output><jupyter_text>3. Train/test split<jupyter_code>df['Observed'] = df['Observed'].astype(float)
df_train, df_test = train_test_split(df, test_size = 0.20, random_state=123)
X_train, y_train = df_train.drop(columns=["Observed"]), df_train["Observed"]
X_test, y_test = df_test.drop(columns=["Observed"]), df_test["Observed"]<jupyter_output><empty_output><jupyter_text>4. EDA<jupyter_code>print("train shape:")
print(df_train.shape)
df_train.dtypes
df_train.head()
df_train.index.min()
df_train.index.max()
from numpy import percentile

quartiles = percentile(df_train['Observed'], [25, 50, 75])
data_min, data_max = df_train['Observed'].min(), df_train['Observed'].max()
print("Observed 5 number summary")
print('Min: %.3f' % data_min)
print('Q1: %.3f' % quartiles[0])
print('Median: %.3f' % quartiles[1])
print('Q3: %.3f' % quartiles[2])
print('Max: %.3f' % data_max)<jupyter_output>Observed 5 number summary
Min: 0.000
Q1: 0.008
Median: 0.163
Q3: 1.613
Max: 192.933<jupyter_text>5. Machine learning model  Ensemble Model<jupyter_code>%%time
from sklearn.datasets import make_regression
from sklearn.ensemble import RandomForestRegressor

# Making ensemble model
ensemble_model = RandomForestRegressor()
# fit the model on the whole dataset
ensemble_model.fit(X_train, y_train)
import math

print(f" Ensemble Train RMSE: {mean_squared_error(y_train, ensemble_model.predict(X_train), squared=False):.2f}")
print(f" Ensemble Test RMSE: {mean_squared_error(y_test, ensemble_model.predict(X_test), squared=False):.2f}")<jupyter_output>Ensemble Train RMSE: 3.15
 Ensemble Test RMSE: 8.85<jupyter_text>Individual Climate Models<jupyter_code>%%time
# Making individual climate model
model_AWI = RandomForestRegressor()
# fit the model on only one column
X_train_awi = X_train[['AWI-ESM-1-1-LR']]
X_test_awi = X_test[['AWI-ESM-1-1-LR']]

model_AWI.fit(X_train_awi, y_train)

print(f" AWI-ESM-1-1-LR Only Train RMSE: {mean_squared_error(y_train, model_AWI.predict(X_train_awi), squared=False):.2f}")
print(f" AWI-ESM-1-1-LR Only Test RMSE: {mean_squared_error(y_test, model_AWI.predict(X_test_awi), squared=False):.2f}")
%%time
# Making individual climate model
model_INM = RandomForestRegressor()
# fit the model on only one column
X_train_inm = X_train[['INM-CM5-0']]
X_test_inm = X_test[['INM-CM5-0']]

model_INM.fit(X_train_inm, y_train)

print(f" INM-CM5-0 Only Train RMSE: {mean_squared_error(y_train, model_INM.predict(X_train_inm), squared=False):.2f}")
print(f" INM-CM5-0 Only Test RMSE: {mean_squared_error(y_test, model_INM.predict(X_test_inm), squared=False):.2f}")
%%time
# Making individual climate model
model_miroc = RandomForestRegressor()
# fit the model on only one column
X_train_miroc = X_train[['MIROC6']]
X_test_miroc= X_test[['MIROC6']]

model_miroc.fit(X_train_miroc, y_train)

print(f" MIROC6 Only Train RMSE: {mean_squared_error(y_train, model_miroc.predict(X_train_miroc), squared=False):.2f}")
print(f" MIROC6 Only Test RMSE: {mean_squared_error(y_test, model_miroc.predict(X_test_miroc), squared=False):.2f}")<jupyter_output>MIROC6 Only Train RMSE: 4.51
 MIROC6 Only Test RMSE: 9.92
CPU times: user 11.4 s, sys: 160 ms, total: 11.6 s
Wall time: 11.6 s<jupyter_text>6. Results - We can see that we achieved a lower test RMSE with the ensemble model (8.85) compared to what we observed with any of the 3 individual models that we tried (10.04 with AWI-ESM-1-1-LR, 10.01 with INM-CM5-0, and 9.94 with MIROC6) - Each individual climate model was bound to have varying levels of accuracy in its prediction. By combining the results of all of them we appear to of been able to build a more accurate machine learning model than by training with any one model on its own.  Part 2:  Preparation for deploying model next week  Complete task 4 from the milestone3 before coming here We’ve found ```n_estimators=100, max_depth=5``` to be the best hyperparameter settings with MLlib (from the task 4 from milestone3), here we then use the same hyperparameters to train a scikit-learn model.<jupyter_code>%%time
best_model = RandomForestRegressor(n_estimators=100, max_depth=5)
best_model.fit(X_train, y_train)
print(f"Train RMSE: {mean_squared_error(y_train, best_model.predict(X_train), squared=False):.2f}")
print(f" Test RMSE: {mean_squared_error(y_test, best_model.predict(X_test), squared=False):.2f}")
# ready to deploy
dump(best_model, "model.joblib")<jupyter_output><empty_output><jupyter_text>***Upload model.joblib to s3. You choose how you want to upload it.***<jupyter_code>import boto3
from botocore.exceptions import NoCredentialsError

ACCESS_KEY = aws_credentials['key']
SECRET_KEY = aws_credentials['secret']


def upload_to_aws(local_file, bucket, s3_file):
    s3 = boto3.client('s3', aws_access_key_id=ACCESS_KEY,
                      aws_secret_access_key=SECRET_KEY)

    try:
        s3.upload_file(local_file, bucket, s3_file)
        print("Upload Successful")
        return True
    except FileNotFoundError:
        print("The file was not found")
        return False
    except NoCredentialsError:
        print("Credentials not available")
        return False
uploaded = upload_to_aws('model.joblib', 'mds-s3-student42', 'output/model.joblib')<jupyter_output>Upload Successful
{'fraction_non_alphanumeric': 0.10805738034589087, 'fraction_numerical': 0.024400053626491485, 'mean_word_length': 5.958955223880597, 'pattern_counts': {'":': 2, '<': 32, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '253320', 'license': 'MIT', 'n_tokens_mistral': 2686, 'n_tokens_neox': 2527, 'n_words': 743, 'path': 'notebooks/Milestone3/Milestone3-Task3.ipynb', 'repo_name': 'UBC-MDS/525_group-20', 'size': '7459'}