We first train an MLP \(M_a\)  jointly with \(F_a\)  for classifying the correct attribute of input \(x\) . The following loss is optimized over \(M_a\)  and \(F_a\) :

La = 1mx X BCE(Ma(Fa(x)), a)  ,

where \(X\)  is the set of training points \(x \in \mathbb {R}^n\)  and \(a\)  is the associated sensitive attribute of \(x\) . \(m = |X|\)  and \(\ell _{BCE}\)  stands for the binary cross-entropy loss. We choose the network architectures, such that \(F_a\)  maps \(x\)  to a low dimensional vector and \(M_a\)  has a low capacity, thus creating a bottleneck [1]} that ensures that \(F_a\)  encodes mostly the information that is relevant to the classification of \(a\) .
