<jupyter_start><jupyter_text>GANITE(Tensorflow): Train and Evaluation This notebook presents the solution for training and evaluating the __GANITE__ algorithm(__Tensoflow__ version) over the [Twins](https://bitbucket.org/mvdschaar/mlforhealthlabpub/src/master/data/twins/) dataset..The implementation of GANITE is adapted in the local `ite` library. For the Unified API version, check [this notebook](https://github.com/bcebere/ite-api/blob/main/notebooks/unified_api_train_evaluation.ipynb).  GANITEEstimating Individualized Treatment Effects(__ITE__) is the task that approximates whether a given treatment influences or determines an outcome([read more](https://www.vanderschaar-lab.com/individualized-treatment-effect-inference/)).[__GANITE__](https://openreview.net/pdf?id=ByKWUeWA-)(Generative Adversarial Nets for inference of Individualized Treatment Effects) is a framework for inferring the ITE using GANs.The implementation demonstrated in this notebook is [here](https://github.com/bcebere/ite-api/tree/main/src/ite/algs/ganite) and is adapted from [this implementation](https://bitbucket.org/mvdschaar/mlforhealthlabpub/src/master/alg/ganite/).  SetupFirst, make sure that all the depends are installed in the current environment.```pip install -r requirements.txtpip install .```Next, we import all the dependencies necessary for the task.<jupyter_code># Double check that we are using the correct interpreter.
import sys
print(sys.executable)

# Disable TF logging
import os 
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
    
# Depends
import ite.algs.ganite.model as alg
import ite.datasets as ds
import ite.utils.tensorflow as utils

from matplotlib import pyplot as plt
import pandas as pd
import tensorflow.compat.v1 as tf<jupyter_output>/home/bcebere/anaconda3/envs/cambridge/bin/python
WARNING:tensorflow:From /home/bcebere/anaconda3/envs/cambridge/lib/python3.8/site-packages/tensorflow/python/compat/v2_compat.py:96: disable_resource_variables (from tensorflow.python.ops.variable_scope) is deprecated and will be removed in a future version.
Instructions for updating:
non-resource variables are not supported in the long term<jupyter_text>Load the DatasetThe example is done using the [Twins](https://bitbucket.org/mvdschaar/mlforhealthlabpub/src/master/data/twins/) dataset.Next, we load the dataset, process the data, and sample a training set and a test set.The logic is implemented [here](https://github.com/bcebere/ite-api/tree/main/src/ite/datasets), and it adapted from the original [GANITE pre-processing implementation](https://bitbucket.org/mvdschaar/mlforhealthlabpub/src/master/alg/ganite/data_preprocessing_ganite.py).<jupyter_code>train_ratio = 0.8
 
dataloader = ds.load("twins", train_ratio)
[Train_X, Train_T, Train_Y, Opt_Train_Y, Test_X, Test_Y] = dataloader


pd.DataFrame(data=Train_X[:5])<jupyter_output><empty_output><jupyter_text>Load the modelNext, we define the model.The constructor supports the following parameters: - `dim`: The number of features in X. - `dim_outcome`: The number of potential outcomes. - `dim_hidden`: hyperparameter for tuning the size of the hidden layer. - `depth`: hyperparameter for the number of hidden layers in the generator and inference blocks. - `num_iterations`: hyperparameter for the number of training epochs. - `alpha`: hyperparameter used for the Generator block loss. - `beta`: hyperparameter used for the ITE block loss. - `num_discr_iterations`: number of iterations executed by the discriminator. - `minibatch_size`: the size of the dataset batches.  The hyperparameters used in this notebook are computed using the [hyperparameter tuning notebook](https://github.com/bcebere/ite-api/blob/main/notebooks/hyperparam_tuning.ipynb).<jupyter_code>dim = len(Train_X[0])
dim_outcome = Test_Y.shape[1]
 
model = alg.Ganite(
    dim,  
    dim_outcome, 
    dim_hidden=7, 
    num_iterations=10000, 
    alpha=5, 
    beta=0.1, 
    minibatch_size=32, 
    num_discr_iterations=9, 
    depth=2,
)

assert model is not None<jupyter_output><empty_output><jupyter_text>Train the model<jupyter_code>metrics = model.train(*dataloader)<jupyter_output>100%|██████████| 10000/10000 [01:12<00:00, 138.48it/s]
100%|██████████| 10000/10000 [00:07<00:00, 1312.45it/s]<jupyter_text>Plot train metrics<jupyter_code>metrics.plot(plt, thresholds = [0.2, 0.25, 0.3, 0.35])

metrics.print()<jupyter_output>Counterfactual Block:
 - Discriminator loss: 0.706 +/- 0.028
 - Generator loss: -3.499 +/- 0.104
ITE Block:
 - Loss: 0.108 +/- 0.034
ITE Block in-sample metrics:
 - sqrt_PEHE: 0.299 +/- 0.004
 - ATE: 0.033 +/- 0.009
 - MSE: 0.095 +/- 0.024
ITE Block out-sample metrics:
 - sqrt_PEHE: 0.287 +/- 0.004
 - ATE: 0.028 +/- 0.009
 - MSE: 0.095 +/- 0.024<jupyter_text>PredictYou can use run inferences on the model and evaluate the output.<jupyter_code>sess = tf.InteractiveSession()

hat_y = model.predict(Test_X)

utils.sqrt_PEHE(hat_y, Test_Y).eval()<jupyter_output><empty_output><jupyter_text>TestWill can run inferences and get metrics directly<jupyter_code>test_metrics = model.test(Test_X, Test_Y)

test_metrics.print()<jupyter_output>sqrt_PHE = 0.281
ATE = 0.022
MSE = 0.078
Top 5 worst mistakes(indices) = [2143  251 2153 1812 1745]
{'fraction_non_alphanumeric': 0.11068408916218293, 'fraction_numerical': 0.03727901614142967, 'mean_word_length': 5.271084337349397, 'pattern_counts': {'":': 0, '<': 27, '<?xml version=': 0, '>': 25, 'https://': 10, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '496337', 'license': 'Apache-2.0', 'n_tokens_mistral': 1916, 'n_tokens_neox': 1701, 'n_words': 455, 'path': 'notebooks/ganite_train_evaluation.ipynb', 'repo_name': 'bcebere/ite-api', 'size': '5204'}