Learning fair representations from the data that encodes biases related to certain sensitive attributes is a widely studied problem in machine learning [1]}.
Some early studies explore improving the fairness of representations by adjusting the raw dataset into a fair one for model training [2]}, [3]}, [4]}.
For example, Calders et al. calders2009building proposed two methods to balance the dataset over different sensitive attributes.
The first one is massaging, which changes the labels of samples with less confident model predictions.
The second one is reweighting, which assigns samples in model training different weights according to the proportion of different sensitive attributes over different classes.
Kamiran et al. kamiran2012data proposed a preferential sampling method that duplicates or removes the samples that are close to the decision boundary based on the attribute distributions.
These dataset modification methods are usually compatible with different tasks and methods.
However, these methods mainly focus on eliminating the effects of biases encoded by training data, and they cannot handle the potential biases and unfairness brought by the representation learning algorithms.
