<jupyter_start><jupyter_text>ERM with DNN under penalty of Equalized Odds We implement here a regular Empirical Risk Minimization (ERM) of a Deep Neural Network (DNN) penalized to enforce an Equalized Odds constraint. More formally, given a dataset of size $n$ consisting of context features $x$, target $y$ and a sensitive information $z$ to protect, we want to solve$$\text{argmin}_{h\in\mathcal{H}}\frac{1}{n}\sum_{i=1}^n \ell(y_i, h(x_i)) + \lambda \chi^2|_1$$where $\ell$ is for instance the MSE and the penalty is$$\chi^2|_1 = \left\lVert\chi^2\left(\hat{\pi}(h(x)|y, z|y), \hat{\pi}(h(x)|y)\otimes\hat{\pi}(z|y)\right)\right\rVert_1$$where $\hat{\pi}$ denotes the empirical density estimated through a Gaussian KDE.  The datasetWe use here the _communities and crimes_ dataset that can be found on the UCI Machine Learning Repository (http://archive.ics.uci.edu/ml/datasets/communities+and+crime). Non-predictive information, such as city name, state... have been removed and the file is at the arff format for ease of loading.<jupyter_code>import sys, os
sys.path.append(os.path.abspath(os.path.join('../..')))
from examples.data_loading import read_dataset
x_train, y_train, z_train, x_test, y_test, z_test = read_dataset(name='crimes', fold=1)
n, d = x_train.shape<jupyter_output><empty_output><jupyter_text>The Deep Neural NetworkWe define a very simple DNN for regression here<jupyter_code>from torch import nn
import torch.nn.functional as F

class NetRegression(nn.Module):
    def __init__(self, input_size, num_classes):
        super(NetRegression, self).__init__()
        size = 50
        self.first = nn.Linear(input_size, size)
        self.fc = nn.Linear(size, size)
        self.last = nn.Linear(size, num_classes)

    def forward(self, x):
        out = F.selu(self.first(x))
        out = F.selu(self.fc(out))
        out = self.last(out)
        return out<jupyter_output><empty_output><jupyter_text>The fairness-inducing regularizerWe implement now the regularizer. The empirical densities $\hat{\pi}$ are estimated using a Gaussian KDE. The L1 functional norm is taken over the values of $y$.$$\chi^2|_1 = \left\lVert\chi^2\left(\hat{\pi}(x|z, y|z), \hat{\pi}(x|z)\otimes\hat{\pi}(y|z)\right)\right\rVert_1$$This used to enforce the conditional independence $X \perp Y \,|\, Z$.Practically, we will want to enforce $\text{prediction} \perp \text{sensitive} \,|\, \text{target}$<jupyter_code>from facl.independence.density_estimation.pytorch_kde import kde
from facl.independence.hgr import chi_2_cond

def chi_squared_l1_kde(X, Y, Z):
    return torch.mean(chi_2_cond(X, Y, Z, kde))<jupyter_output><empty_output><jupyter_text>The fairness-penalized ERMWe now implement the full learning loop. The regression loss used is the quadratic loss with a L2 regularization and the fairness-inducing penalty.<jupyter_code>import torch
import numpy as np
import torch.utils.data as data_utils

def regularized_learning(x_train, y_train, z_train, model, fairness_penalty, lr=1e-5, num_epochs=10):
    # wrap dataset in torch tensors
    Y = torch.tensor(y_train.astype(np.float32))
    X = torch.tensor(x_train.astype(np.float32))
    Z = torch.tensor(z_train.astype(np.float32))
    dataset = data_utils.TensorDataset(X, Y, Z)
    dataset_loader = data_utils.DataLoader(dataset=dataset, batch_size=200, shuffle=True)

    # mse regression objective
    data_fitting_loss = nn.MSELoss()

    # stochastic optimizer
    optimizer = torch.optim.Adam(model.parameters(), lr=lr, weight_decay=0.01)

    for j in range(num_epochs):
        for i, (x, y, z) in enumerate(dataset_loader):
            def closure():
                optimizer.zero_grad()
                outputs = model(x).flatten()
                loss = data_fitting_loss(outputs, y)                
                loss += fairness_penalty(outputs, z, y)
                loss.backward()
                return loss

            optimizer.step(closure)
    return model<jupyter_output><empty_output><jupyter_text>EvaluationFor the evaluation on the test set, we compute two metrics: the MSE (accuracy) and HGR$|_\infty$ (fairness).<jupyter_code>from facl.independence.hgr import hgr_cond

def evaluate(model, x, y, z):
    Y = torch.tensor(y.astype(np.float32))
    Z = torch.Tensor(z.astype(np.float32))
    X = torch.tensor(x.astype(np.float32))

    prediction = model(X).detach().flatten()
    loss = nn.MSELoss()(prediction, Y)
    hgr_infty = np.max(hgr_cond(prediction, Z, Y, kde))
    return loss.item(), hgr_infty<jupyter_output><empty_output><jupyter_text>Running everything together<jupyter_code>model = NetRegression(d, 1)

num_epochs = 20
lr = 1e-5

# $\chi^2|_1$
penalty_coefficient = 1.0
penalty = chi_squared_l1_kde

model = regularized_learning(x_train, y_train, z_train, model=model, fairness_penalty=penalty, lr=lr, \
                             num_epochs=num_epochs)

mse, hgr_infty = evaluate(model, x_test, y_test, z_test)
print("MSE:{} HGR_infty:{}".format(mse, hgr_infty))<jupyter_output>torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([194])
torch.Size([194])
torch.Size([194])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([200])
torch.Size([194])
torch.Size([194])
torch.Size([194])
torch.Size([200])
torch.Size[...]
{'fraction_non_alphanumeric': 0.1485247541256876, 'fraction_numerical': 0.03617269544924154, 'mean_word_length': 4.504587155963303, 'pattern_counts': {'":': 0, '<': 24, '<?xml version=': 0, '>': 24, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '11823', 'license': 'MIT', 'n_tokens_mistral': 2262, 'n_tokens_neox': 2122, 'n_words': 484, 'path': 'others/third_party/fairness_aware_learning/examples/continuous_fairness_aware_learning/equalized_odds_dnn.ipynb', 'repo_name': 'altosaar/fair_dummies', 'size': '5999'}