<jupyter_start><jupyter_text>Set WD<jupyter_code>import os
os.getcwd()
os.chdir('/home/jovyan/work/adv_dsi_lab_5/')<jupyter_output><empty_output><jupyter_text>Load and Explore Data Task: Launch the magic commands for auto-relaoding external modules<jupyter_code>%load_ext autoreload
%autoreload 2
import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>Create a variable called file_url containing th url to the raw dataset<jupyter_code>file_url = 'https://code.datasciencedojo.com/datasciencedojo/datasets/raw/master/Beijing%20PM2.5/PRSA_data_2010.1.1-2014.12.31.csv'<jupyter_output><empty_output><jupyter_text>Load the data in a dataframe called df<jupyter_code>df = pd.read_csv(file_url)
df.head()
df.shape
df.info()
df.describe()<jupyter_output><empty_output><jupyter_text>Save the dataframe locally in the data/raw folder<jupyter_code>df.to_csv('data/raw/pollution.csv', index=False)<jupyter_output><empty_output><jupyter_text>3. Prepare Data Create a copy of df and save it into a variable called df_cleaned. Remove the missing values from the target variable pm2.5<jupyter_code>df_cleaned = df.copy()
df_cleaned.drop('No', axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Remove the missing values from the target variable pm2.5<jupyter_code>df_cleaned.dropna(inplace=True)<jupyter_output><empty_output><jupyter_text>Reset the indexes of the dataframe<jupyter_code>df_cleaned.reset_index(drop=True, inplace=True)
from sklearn.preprocessing import StandardScaler, OneHotEncoder<jupyter_output><empty_output><jupyter_text>Create a list called num_cols that contains year, DEWP, TEMP, PRES, Iws, Is, Ir<jupyter_code>num_cols = ['year', 'DEWP', 'TEMP', 'PRES', 'Iws', 'Is', 'Ir']<jupyter_output><empty_output><jupyter_text>Instantiate a StandardScaler and called it sc<jupyter_code>sc = StandardScaler()<jupyter_output><empty_output><jupyter_text>Fit and transform the numeric feature of df_cleaned and replace the data into it<jupyter_code>df_cleaned[num_cols] = sc.fit_transform(df_cleaned[num_cols])<jupyter_output><empty_output><jupyter_text>Create a list called cat_cols that contains Gender<jupyter_code>cat_cols = ['month', 'day', 'hour', 'cbwd']<jupyter_output><empty_output><jupyter_text>Instantiate a OneHotEncoder and call it ohe<jupyter_code>ohe = OneHotEncoder(sparse=False)<jupyter_output><empty_output><jupyter_text>Perform One-Hot encoding on cat_cols and save them into a dataframe called X_cat<jupyter_code>X_cat = pd.DataFrame(ohe.fit_transform(df_cleaned[cat_cols]))<jupyter_output><empty_output><jupyter_text>Extract the feature names from ohe and replace the names of the columns of the X_cat<jupyter_code>X_cat.columns = ohe.get_feature_names(cat_cols)<jupyter_output><empty_output><jupyter_text>Drop the original columns of cat_cols from df_cleaned<jupyter_code>df_cleaned.drop(cat_cols, axis=1, inplace=True)<jupyter_output><empty_output><jupyter_text>Concatenate df_cleaned with X_cat and save the result to a variable called X<jupyter_code>X = pd.concat([df_cleaned, X_cat], axis=1)
X.head()<jupyter_output><empty_output><jupyter_text>Import split_sets_by_time and save_sets from src.data.sets<jupyter_code>from src.data.sets import split_sets_by_time, save_sets<jupyter_output><empty_output><jupyter_text>Split the data into training and testing sets with 80-20 ratio<jupyter_code>X_train, y_train, X_val, y_val, X_test, y_test = split_sets_by_time(X, target_col='pm2.5', test_ratio=0.2)
save_sets(X_train=X_train, y_train=y_train, X_val=X_val, y_val=y_val, X_test=X_test, y_test=y_test, path='/home/jovyan/work/adv_dsi_lab_5/data/processed/beijing_pollution/')<jupyter_output><empty_output><jupyter_text>4. Baseline Model Import NullModel from src.models.null<jupyter_code>from src.models.null import NullModel<jupyter_output><empty_output><jupyter_text>Instantiate a NullModel and call .fit_predict() on the training target to extract your predictions into a variable called y_base<jupyter_code>baseline_model = NullModel()
y_base = baseline_model.fit_predict(y_train)<jupyter_output><empty_output><jupyter_text>Import print_reg_perf from src.models.performance<jupyter_code>from src.models.performance import print_reg_perf
print_reg_perf(y_base, y_train, set_name='Training')<jupyter_output>RMSE Training: 92.82545840756482
MAE Training: 69.67082209440568<jupyter_text>5. Define Architecture<jupyter_code>import torch
import torch.nn as nn
import torch.nn.functional as F<jupyter_output><empty_output><jupyter_text>Create in src/models/pytorch.py a class called PytorchRegression that inherits from nn.Module with:- num_features as input parameter- attributes:    -layer_1: fully-connected layer with 128 neurons    -layer_out: fully-connected layer with 1 neurons-methods:    -forward() with inputs as input parameter, perform ReLU and DropOut on the fully-connected layer followed by the output layer<jupyter_code>from src.models.pytorch import PytorchRegression<jupyter_output><empty_output><jupyter_text>Instantiate PytorchRegression with the correct number of input feature and save it into a variable called model<jupyter_code>model = PytorchRegression(X_train.shape[1])<jupyter_output><empty_output><jupyter_text>def get_device():    if torch.cuda.is_available():        device = torch.device('cuda:0')    else:        device = torch.device('cpu')  don't have GPU     return device Set model to use the device available<jupyter_code>from src.models.pytorch import get_device

device = get_device()
model.to(device)<jupyter_output><empty_output><jupyter_text>Print model architecture<jupyter_code>print(model)<jupyter_output>PytorchRegression(
  (layer_1): Linear(in_features=78, out_features=128, bias=True)
  (layer_out): Linear(in_features=128, out_features=1, bias=True)
)<jupyter_text>6. Create Data Loader Import Dataset and DataLoader from torch.utils.data<jupyter_code>from torch.utils.data import Dataset, DataLoader<jupyter_output><empty_output><jupyter_text>Create in src/models/pytorch.pya class called PytorchDataset that inherits from torch.utils.data.Dataset with:X ans y as input parametersattributes:X_tensor: X converted to Pytorch tensory_tensor: y converted to Pytorch tensormethods:__getitem__(index) Return features and target for a given index__len__ Return the number of observationsto_tensor(data) Convert Pandas Series to Pytorch tensor Import this class from src/models/pytorch and convert all sets to PytorchDataset<jupyter_code>from src.models.pytorch import PytorchDataset
train_dataset = PytorchDataset(X=X_train, y=y_train)
val_dataset = PytorchDataset(X=X_val, y=y_val)
test_dataset = PytorchDataset(X=X_test, y=y_test)<jupyter_output><empty_output><jupyter_text>Import DataLoader from torch.utils.data<jupyter_code>from torch.utils.data import DataLoader<jupyter_output><empty_output><jupyter_text>7. Train Model Instantiate a nn.MSELoss() and save it into a variable called criterion<jupyter_code>criterion = nn.MSELoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)
from src.models.train_regression_model import train_regression
from src.models.test_regression_model import test_regression<jupyter_output><empty_output><jupyter_text>Create 2 variables called N_EPOCHS and BATCH_SIZE that will take respectively 5 and 32 as values<jupyter_code>N_EPOCHS = 5
BATCH_SIZE = 32<jupyter_output><empty_output><jupyter_text>Create a for loop that will iterate through the specified number of epochs and will train the model with the training set and assess the performance on the validation set and print their scores<jupyter_code>for epoch in range(N_EPOCHS):
    train_loss, train_rmse = train_regression(train_dataset, model=model, criterion=criterion, optimizer=optimizer, batch_size=BATCH_SIZE, device=device)
    valid_loss, valid_rmse = test_regression(val_dataset, model=model, criterion=criterion, batch_size=BATCH_SIZE, device=device)

    print(f'Epoch: {epoch}')
    print(f'\t(train)\tLoss: {train_loss:.4f}\t|\tRMSE: {train_rmse:.1f}')
    print(f'\t(valid)\tLoss: {valid_loss:.4f}\t|\tRMSE: {valid_rmse:.1f}')<jupyter_output>Epoch: 0
	(train)	Loss: 272.4183	|	RMSE: 16.5
	(valid)	Loss: 236.9014	|	RMSE: 15.4
Epoch: 1
	(train)	Loss: 272.2880	|	RMSE: 16.5
	(valid)	Loss: 237.0541	|	RMSE: 15.4
Epoch: 2
	(train)	Loss: 272.2193	|	RMSE: 16.5
	(valid)	Loss: 237.9660	|	RMSE: 15.4
Epoch: 3
	(train)	Loss: 272.0095	|	RMSE: 16.5
	(valid)	Loss: 237.8078	|	RMSE: 15.4
Epoch: 4
	(train)	Loss: 272.1025	|	RMSE: 16.5
	(valid)	Loss: 237.0974	|	RMSE: 15.4<jupyter_text>Save the model into the models folder<jupyter_code>torch.save(model, "models/pytorch_reg_pm2_5.pt")<jupyter_output><empty_output><jupyter_text>8.   Assess Performance Assess the model performance on the testing set and print its scores<jupyter_code>test_loss, test_rmse = test_regression(test_dataset, model=model, criterion=criterion, batch_size=BATCH_SIZE, device=device)
print(f'\tLoss: {test_loss:.4f}\t|\tRMSE: {test_rmse:.1f}')<jupyter_output>Loss: 277.0515	|	RMSE: 16.6
{'fraction_non_alphanumeric': 0.09957627118644068, 'fraction_numerical': 0.02508920606601249, 'mean_word_length': 8.105583756345178, 'pattern_counts': {'":': 0, '<': 137, '<?xml version=': 0, '>': 137, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '35', 'dir': 'jupyter-structured-clean-dedup', 'id': '273557', 'license': 'FTL', 'n_tokens_mistral': 3413, 'n_tokens_neox': 3238, 'n_words': 693, 'path': 'notebooks/1_pytorch_regression.ipynb', 'repo_name': 'Kallikrates/adv_dsi_lab_5', 'size': '8968'}