<jupyter_start><jupyter_text>Clearbox Wrapper TutorialClearbox Wrapper is a Python library to package and save a ML model.We'll use the popular Diabetes Hospital Readmissions dataset and build a Pytorch classifier on it.This is a typical case: before feeding the data to the model, we need to preprocess them. Pre-processing code is usually written as a separate element wrt to the model, during the development phase. We want to wrap and save the preprocessing along with the model so to have a pipeline Processing+Model ready to take unprocessed data, process them and make predictions.We can do that with Clearbox Wrapper, but all the preprocessing code must be wrapped in a single function. In this way, we can pass the function to the _save_model_ method. Install and import required libraries<jupyter_code>%%capture 
!pip install pandas
!pip install numpy
!pip install scikit-learn
!pip install torch

!pip install clearbox-wrapper==0.3.10
import pandas as pd
import numpy as np

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.impute import SimpleImputer
import sklearn.metrics as metrics


import torch
import torch.nn as nn

import clearbox_wrapper as cbw<jupyter_output><empty_output><jupyter_text>DatasetsWe have two different csv files for the training and test set.<jupyter_code>hospital_training_csv_path = 'hospital_readmissions_training.csv'
hospital_test_csv_path = 'hospital_readmissions_test.csv'
hospital_training = pd.read_csv(hospital_training_csv_path)
hospital_test = pd.read_csv(hospital_test_csv_path)
target_column = 'readmitted'
y_train = hospital_training[target_column]
X_train = hospital_training.drop(target_column, axis=1)
y_test = hospital_test[target_column]
X_test = hospital_test.drop(target_column, axis=1)
X_train.info()
X_test.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 12290 entries, 0 to 12289
Data columns (total 40 columns):
 #   Column                    Non-Null Count  Dtype 
---  ------                    --------------  ----- 
 0   age                       12290 non-null  int64 
 1   gender                    12290 non-null  object
 2   race                      12290 non-null  object
 3   diagnosis                 12290 non-null  object
 4   time_in_hospital          12290 non-null  int64 
 5   num_procedures            12290 non-null  int64 
 6   num_lab_procedures        12290 non-null  int64 
 7   num_medications           12290 non-null  int64 
 8   num_inpatient             12290 non-null  int64 
 9   num_outpatient            12290 non-null  int64 
 10  num_emergency             12290 non-null  int64 
 11  num_diagnoses             12290 non-null  int64 
 12  service_utilization       12290 non-null  int64 
 13  num_change                12290 non-null  int64 
 14  num_drugs             [...]<jupyter_text>Create a preprocessing functionThe data need to be preprocessed before be passed as input to the model. You can use your own custom code for the preprocessing, just remember to wrap all of it in a single function.Here, we create a preprocessing pipeline for the X using sklearn ColumnTransformer and Pipeline, then we fit it on the training X. The resulting x-processing is already a single function, so ready to be passed to the wrapper:<jupyter_code>ordinal_features = X_train.select_dtypes(include="number").columns
categorical_features = X_train.select_dtypes(include="object").columns
ordinal_transformer = Pipeline(
    steps=[
        ("imputer", SimpleImputer(strategy="median")),
        ("scaler", StandardScaler()),
    ]
)
categorical_transformer = Pipeline(
    steps=[
        ("imputer", SimpleImputer(strategy="most_frequent")),
        ("onehot", OneHotEncoder(handle_unknown="ignore")),
    ]
)
x_processing = ColumnTransformer(
    transformers=[
        ("ord", ordinal_transformer, ordinal_features),
        ("cat", categorical_transformer, categorical_features),
    ]
)
x_processing.fit(X_train)<jupyter_output><empty_output><jupyter_text>Then, we wrap the processing into a function:<jupyter_code>def hospital_preprocessing(x_data):
    processed_data = x_processing.transform(x_data)
    return processed_data<jupyter_output><empty_output><jupyter_text>As usual we encode the Y labels through a simple OneHotEncoder<jupyter_code>y_processing = OneHotEncoder()
y_processing.fit(y_train.values.reshape(-1, 1))<jupyter_output><empty_output><jupyter_text>Create and train the modelWe build a Pytorch network setting some basic parameters...<jupyter_code>num_epochs = 35
learning_rate = 0.0001
size_hidden1 = 50
size_hidden2 = 30
size_hidden3 = 10
size_hidden4 = 2
class HospitalModel(nn.Module):
    def __init__(self, input_shape):
        super().__init__()
        self.lin1 = nn.Linear(input_shape, size_hidden1)
        self.relu1 = nn.ReLU()
        self.lin2 = nn.Linear(size_hidden1, size_hidden2)
        self.relu2 = nn.ReLU()
        self.lin3 = nn.Linear(size_hidden2, size_hidden3)
        self.relu3 = nn.ReLU()
        self.lin4 = nn.Linear(size_hidden3, size_hidden4)
        self.softmax = nn.Softmax(dim=1)

    def forward(self, input):
        return self.softmax(
            self.lin4(
                self.relu3(
                    self.lin3(self.relu2(self.lin2(self.relu1(self.lin1(input)))))
                )
            )
        )<jupyter_output><empty_output><jupyter_text>...add the training function:<jupyter_code>def train(model_inp, x_train, y_train, num_epochs=num_epochs):
    datasets = torch.utils.data.TensorDataset(x_train, y_train)
    train_iter = torch.utils.data.DataLoader(datasets, batch_size=500, shuffle=True)
    criterion = nn.BCELoss()
    optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)
    for epoch in range(num_epochs):  # loop over the dataset multiple times
        running_loss = 0.0
        for inputs, labels in train_iter:
            # forward pass
            outputs = model_inp(inputs)
            # defining loss
            loss = criterion(outputs, labels)
            # zero the parameter gradients
            optimizer.zero_grad()
            # computing gradients
            loss.backward()
            # accumulating running loss
            running_loss += loss.item()
            # updated weights based on computed gradients
            optimizer.step()
        if (epoch+1) % 5 == 0:
            print(
                "Epoch [%d]/[%d] running accumulative loss across all batches: %.3f"
                % (epoch + 1, num_epochs, running_loss)
            )<jupyter_output><empty_output><jupyter_text>So we proceed to pre-process the training and test data:<jupyter_code>X_train_processed = hospital_preprocessing(X_train)
X_train_processed_tensor = torch.Tensor(X_train_processed)
X_test_processed = hospital_preprocessing(X_test)
X_test_processed_tensor = torch.Tensor(X_test_processed)<jupyter_output><empty_output><jupyter_text>Then, we encode the y training and test data and convert them in the Pytorch Tensor format:<jupyter_code>y_train_processed = y_processing.transform(y_train.values.reshape(-1, 1))
y_train_processed_tensor = torch.Tensor(y_train_processed.todense())
y_test_processed = y_processing.transform(y_test.values.reshape(-1, 1))
y_test_processed_tensor = torch.Tensor(y_test_processed.todense())<jupyter_output><empty_output><jupyter_text>...and finally create a model instance and fit it on the resulting data:<jupyter_code>model = HospitalModel(X_train_processed_tensor.shape[1])
model.train()
train(model, X_train_processed_tensor, y_train_processed_tensor)<jupyter_output>/home/andrea/clearbox_repos/clearbox-wrapper/.venv/lib/python3.8/site-packages/torch/autograd/__init__.py:130: UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at  /pytorch/c10/cuda/CUDAFunctions.cpp:100.)
  Variable._execution_engine.run_backward(
Epoch [5]/[35] running accumulative loss across all batches: 99.050
Epoch [10]/[35] running accumulative loss across all batches: 86.363
Epoch [15]/[35] running accumulative loss across all batches: 79.273
Epoch [20]/[35] running accumulative loss across all batches: 76.271
Epoch [25]/[35] running accumulative loss across all batches: 74.820
Epoch [30]/[35] running accumulative loss across all batches: 74.013
Epoch [35]/[35] running accumulative loss across all batches: 73.449<jupyter_text>We show some metrics on the training set...<jupyter_code>model.eval()
training_predictions = model(X_train_processed_tensor)
training_predictions = np.argmax(training_predictions.detach().numpy(), axis=-1)
print('Training Metrics Report:\n', metrics.classification_report(np.argmax(y_train_processed, axis=-1), training_predictions))<jupyter_output>Training Metrics Report:
               precision    recall  f1-score   support

           0       0.83      0.88      0.85     55319
           1       0.87      0.82      0.84     55293

    accuracy                           0.85    110612
   macro avg       0.85      0.85      0.85    110612
weighted avg       0.85      0.85      0.85    110612<jupyter_text>...and on the test set:<jupyter_code>test_predictions = model(X_test_processed_tensor)
test_predictions = np.argmax(test_predictions.detach().numpy(), axis=-1)
print('Test Metrics Report:\n', metrics.classification_report(np.argmax(y_test_processed, axis=-1), test_predictions))<jupyter_output>Test Metrics Report:
               precision    recall  f1-score   support

           0       0.82      0.88      0.85      6132
           1       0.87      0.81      0.84      6158

    accuracy                           0.84     12290
   macro avg       0.85      0.84      0.84     12290
weighted avg       0.85      0.84      0.84     12290<jupyter_text>Wrap and Save the ModelFinally, we use Clearbox Wrapper to wrap and save the model and the preprocessor as a zipped folder in a specified path. The model dependency (torch) and its version it is detected automatically by CBW and added to the requirements saved into the resulting folder. But (**IMPORTANT**) you need to pass as a parameter the additional dependencies required for the preprocessing as a list. We just need to add Scikit-Learn in this case.We pass the training dataset (X train) to `save_model` as well, in order to generate a Preprocessing and Model Signature (the signature represents input and (optionally) outputs as data frames with (optionally) named columns and data type).<jupyter_code>wrapped_model_path = 'hospital_wrapped_model_v0.3.10'
processing_dependencies = ["scikit-learn==0.23.2"]
cbw.save_model(wrapped_model_path, model, preprocessing=hospital_preprocessing, input_data=X_train, additional_deps=processing_dependencies)<jupyter_output><empty_output><jupyter_text>Unzip and load the modelThe following cells are not necessary for the final users, the zip created should be uploaded to our SAAS as it is. But here we want to show how to load a saved model and compare it to the original one.<jupyter_code>import zipfile
zipped_model_path = 'hospital_wrapped_model_v0.3.10.zip'
unzipped_model_path = 'hospital_wrapped_model_v0.3.10_unzipped'
with zipfile.ZipFile(zipped_model_path, 'r') as zip_ref:
    zip_ref.extractall(unzipped_model_path)
loaded_model = cbw.load_model(unzipped_model_path)
original_model_predictions = model(X_test_processed_tensor).detach().numpy()
loaded_model_predictions = loaded_model.predict(X_test)
np.testing.assert_array_equal(original_model_predictions, loaded_model_predictions)<jupyter_output><empty_output><jupyter_text>Remove all generated files and directory<jupyter_code>import os
import shutil
if os.path.exists(zipped_model_path):
        os.remove(zipped_model_path)
if os.path.exists(unzipped_model_path):
        shutil.rmtree(unzipped_model_path)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07983614780137101, 'fraction_numerical': 0.03469319511787326, 'mean_word_length': 3.7154119038234135, 'pattern_counts': {'":': 0, '<': 58, '<?xml version=': 0, '>': 58, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 5, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '346297', 'license': 'Apache-2.0', 'n_tokens_mistral': 3925, 'n_tokens_neox': 3594, 'n_words': 1009, 'path': 'examples/5_hospital_preprocessing_pytorch/5_Clearbox_Wrapper_Hospital_Pytorch.ipynb', 'repo_name': 'Clearbox-AI/clearbox-wrapper', 'size': '11962'}