Computer Science
Computer Science-2021.gz:143338
Computer Science
bd5041238e01d6c01b68f96b5cdf8ca2bd24f0ca
Removing biased data to improve fairness and accuracy

Machine learning systems are often trained using data collected from historical decisions. If past decisions were biased, then automated systems that learn from historical data will also be biased. We propose a black-box approach to identify and remove biased training data. Machine learning models trained on such debiased data (a subset of the original training data) have low individual discrimination, often 0%. These models also have greater accuracy and lower statistical disparity than models trained on the full historical data. We evaluated our methodology in experiments using 6 real-world datasets. Our approach outperformed seven previous approaches in terms of individual discrimination and accuracy.


INTRODUCTION
Automated decisions can be faster, cheaper, and less subjective than manual ones. In order to automate decisions, an organization can train a machine learning model on historical decisions or other manually-labeled data and use it to make decisions in the future. However, if the training data is biased, that bias will be reflected in the model's decisions [22,40]. This problem has even been noted by the US White House [91].
Bias in models can have far-reaching societal consequences, like worsening wealth inequality [46], difference in employment rate across genders [24,80], and difference in incarceration rates across races [63]. Discrimination has been reported in machine-produced decisions for real-life scenarios like parole [53], credit cards [69], hiring [24], and predictive policing [41]. These problems are exacerbated by humans' unwarranted confidence in machine-produced decisions: people generally deem such decisions fair [25]. a Data bias happens mostly due to two phenomena: label bias and selection bias. Label bias occurs when the training labels (which are mostly generated manually) are afflicted by human bias. For example, loan applications and job applications from minority communities have been more frequently denied [42,103,115]. Training on historical data would perpetuate that injustice. Selection bias occurs when selecting samples from a demographic group inadvertently introduces undesired correlations between the features pertaining to that demographic group and the training labels [12,113,120], e.g. in the selected subsample for a group, most of their loan requests were denied. The propagation of bias has raised significant concerns related to use of machine learning models in critical decision making like the ones mentioned above. Bias in machine learning systems is undesirable because it can produce unfair decisions [23,24], because biased decisions are less effective and less profitable [4,67], and because bias attracts lawsuits and widespread criticism [69,93]. Biased decisions can be challenged on the basis of disparate treatment and disparate impact laws in the US [6,52,99,102,118]. Similar laws exist in other countries [5]. Laws define sensitive attributes (e.g., race, sex, religion) that are illegal to use as the basis of any decision.
We use both the common statistical disparity, and also individual discrimination as metrics to measure discrimination, in particular for machine learning classifiers. (Other definitions of fairness exist; section 5 justifies our choice.) A model is individually fair [33] if it yields similar predictions for similar individuals. A similar pair consists of two individuals ⟨ , ⟩ such that ≠ and is similar to . Similarity among the individuals and among the predictions is measured by defining a distance function in the input space and the output space, respectively. A similar pair of individuals are two individuals with distance lower than a specified threshold. Ideally, for ensuring fairness, the input space similarity metric should not take sensitive features into account, so two individuals who differ only in the sensitive features should always be similar. A discriminatory pair is a similar pair such that the model makes dissimilar predictions for the two similar individuals. The individual discrimination of a model is the proportion of similar pairs of individuals that receive dissimilar predictions, i.e., the percentage of discriminatory pairs [3,39,107]. An estimate of individual discrimination uses a pool of similar pairs, which may be synthetically generated or sampled from the training data. To be fair, a model's individual discrimination should be zero or close to it.
Previous approaches to reducing discrimination (see section 5) modify the training data's features or labels (pre-processing), or add fairness regularizers while training (in-processing), or perform post-hoc corrections in the model's predictions (post-processing).
Our approach is a pre-processing approach that identifies and removes biased datapoints from the training data, leaving all other datapoints unchanged. We conjectre that some training data is more biased than the rest and consequently has more influence on the predictions of a learned model. Influence functions [68] measure the influence of a training datapoint on a particular prediction. Our approach (1) generates discriminatory pairs (similar individuals who receive dissimilar predictions) and (2) uses influence functions to sort the training datapoints in order of most to least influential for the dissimilar predictions received by the discriminatory pairs. We hypothesize that the datapoints that are most responsible for the dissimilar predictions are the most biased datapoints. Removing the most influential datapoints yields a debiased dataset. A model trained on the debiased dataset is fairer with less individual discrimination -often 0% -than a model trained on the full dataset. Our approach works for black-box classification models (for which only train and predict functions are available) [17,86], and therefore proprietary models can be debiased as long as their training data is accessible. We performed 8 experiments using 6 real-world datasets (some datasets contain more than one sensitive attribute). Our approach outperforms seven previous approaches in terms of individual discrimination and accuracy and is near the average in terms of statistical disparity.
In summary, our contributions are: • We propose a novel black-box approach for improving individual fairness: identify and remove biased decisions in historical data. • In two sets of experiments using 6 real-world datasets, the classifiers trained on debiased datasets exhibit nearly 0% individual discrimination. Our approach outperforms seven previous approaches in terms of individual discrimination and accuracy, and it always improves (reduces) statistical disparity. • To the best of our knowledge, we are the first to empirically demonstrate an increase in test accuracy (better generalization) in a supervised learning setup with real datasets while reducing discrimination, compared to the case when no fairness technique is used [12,113]. • Our implementation and experimental scripts are open source.

MOTIVATING EXAMPLE
Suppose that a bank wishes to automate the process of loan approval. The bank could train a machine learning model on historical loan decisions, then use the model to improve speed, reduce costs, and reduce subjectivity. The financial sector is a prominent user of machine learning [26,104]. Table 1 shows a hypothetical dataset consisting of historical loan decisions. The bank collects 3 features from each applicant: income, wealth, and race. As shown in the table, income and wealth are numeric features with values lying between 0 and 1, after normalization. Race is a binary feature with 'white' and 'black' as the two possible values. The outcome is also binary with '1' indicating that a loan was offered and '0' indicating a denial.
Loan approvals should ideally not depend on the applicant's race, but past loan approvers might have been consciously or unconsciously biased. In fact, that was the case at this bank: #2 was a biased decision in which a black person with income in high range (0.9) was denied a loan, whereas all white people with high range income (> 0.66) were given a loan. A classifier trained on a dataset that contains biased datapoints is likely to exhibit individual discrimination.
The bank would like to train a classifier using the unbiased parts of the historical data. Most of the decisions were probably sound (otherwise, the bank would have been out-competed by its rivals). It would be too expensive to filter biased decisions or create a new dataset manually, and those manual activities would also be prone to conscious and unconscious bias.

Measuring individual discrimination
Estimating a model's individual discrimination requires the distance functions for the input and output space, and a pool of similar pairs. We consider two individuals similar if their income and wealth are the same, regardless of their race. We consider two decisions similar if they are the same decision. We randomly generate 700 similar pairs (see Section 3 for details). This is 100 times as large as the training dataset; the value "100" is arbitrary.
We trained a model on the dataset of table 1 and evaluated it on the 1400 individuals who form the 700 pairs of similar individuals. The model predicted different outcomes for 26% of the similar pairs (181 discriminatory pairs out of the 700 similar pairs).

Finding biased datapoints
Our goal is to find biased training datapoints, such that removing them reduces the model's individual discrimination. This also improves the model's accuracy and statistical parity, which is discussed in Section 4. Our approach contains three high-level steps: (1) Find unfair decisions made by a learned model. In particular, first generate a pool of pairs of similar individuals and find the discriminating pairs within them. Then, for each discriminating pair, heuristically determine which of the two individuals was not treated fairly. (2) Rank the training datapoints according to their contribution to the unfair decisions. (3) Remove some of the highest-ranked training datapoints and retrain a model with a lower individual discrimination.
Identify unfair decisions. There are 181 discriminatory pairspairs of similar individuals with dissimilar outcomes. One individual from each pair has been treated unfairly. The biased treatment might be in their favor (getting an undeserved loan) or against them (denied a loan they deserved). Our heuristic is that the individual whose classification confidence (i.e., the probability with which an individual is classified) is lower is the one who was treated unfairly. Note that these are the individuals in the generated pool of similar individuals, not in the training dataset.
Identify training datapoints responsible for unfair decisions. We use influence functions [68] to find the training datapoints that were most responsible for producing different predictions for the discriminatory pairs. An influence function sorts the training datapoints of a model in order of most to least responsible for a single prediction from the model. An influence function can also sort the datapoints for a set of predictions (the 181 unfair decisions, in    We hypothesize that influence functions rank the training datapoints in order of most to least biased datapoints. Note that, the discriminatory pairs are only used to measure discrimination and to sort datapoints in the original training dataset; the discriminatory pairs themselves never occur in the training data.
Remove biased datapoints. When #2 is removed, and the model is retrained, it results in only 1 discriminatory pair out of the same 700 similar pairs of individuals, i.e., 0.14% remaining individual discrimination. When both #2 and #7 are removed, the discrimination is 16%. Therefore, removing only #2 is the local minimum for discrimination. Our algorithm returns the dataset with #2 removed. This reduced dataset is called the "debiased dataset". Note that our approach removes biased datapoints from the training data. The similar pairs are synthetically generated and are only used to estimate individual discrimination and identify biased training points. The similar pairs can not be removed since they don't occur in the training dataset.
The bank should train its model on the fair decisions in the debiased dataset, rather than on the whole dataset. We choose to remove the biased datapoint and not modify its label because the bias could arise either due to labeling bias or due to selection bias. Note that our approach can remove a datapoint belonging to any demographic group, not only the disadvantaged group as happened in this hypothetical dataset.
All the steps of our approach are shown in Figure 1.
Our approach requires access to sensitive attribute. Veale and Binns [108], Williams et al. [116] have argued about the necessity of having access to the sensitive features in order to identify and address bias in automated models. After identifying and removing the biased decisions, the bank can train their model by omitting the sensitive feature(s) (race in this case) to avoid disparate treatment [52].
Our approach empowers the bank to make less biased decisions in the future. (As with any automated decision-making process, the bank should include avenues for challenge and redress.) This avoids violating anti-discrimination laws and is the morally right thing to do.

ALGORITHM
Algorithm 1 sorts a dataset in the order of most biased to least biased datapoint. It has four main parts.
First, line 2 generates a pool PSI of pairs of similar individuals using GenerateSimilarPairs (line 11). We arbitrarily choose the size of the pool to be 100 times the size of the dataset. A larger pool leads to a better estimate of individual discrimination. The individuals and their similar counterparts are automatically generated, sampling uniformly at random from the feature space that is defined by the original training dataset. For a randomly sampled individual A1, SimilarIndividual (line 15) generates a similar individual A2 whose distance from A1 is less than the specified threshold , which is a user-provided parameter that determines whether two individuals are similar. For the input space similarity condition in Section 2, is 0. For example, random sampling for income, wealth, and race in the hypothetical dataset could generate an individual A1 with the feature vector ⟨0.9, 0.5, ℎ ⟩. According to the input space similarity condition, a similar individual for A1 must have the same income and wealth (these are the non-sensitive features). While generating similar individuals, we enforce changing the value of the sensitive attribute (race in this case) to generate similar individuals in different demographic groups. Therefore, A2 // Randomly generates pairs of similar individuals 11 Function GenerateSimilarPairs(D, S, ) 12 SimilarPairs ← {} return SimilarPairs // Ranks datapoints in D responsible for discrimination, sorted in decreasing order of influence 18 Function RankByInfluence(InfluenceSet, D) // See [68] for implementation has the feature vector ⟨0.9, 0.5, ⟩. These pairs of similar individuals allow us to estimate the individual discrimination of a model because we can compare their predictions, which should be similar as well. Note that generating pairs of similar individuals based on a given , as opposed to finding pairs of similar individuals in the training dataset, allows us to reliably generate a large pool of pairs. Second, line 3 determines the discriminatory pairs: the pairs of similar individuals who received dissimilar model predictions (a subset of the pool PSI). In the case of binary classification, different labels are considered dissimilar.
Third, lines 4 to 9 determine, for each discriminatory pair, which individual was misclassified due to model bias. Internally, a classifier computes, for each outcome class, the probability of an individual belonging to that class, and the predicted outcome for that individual is the class with the highest probability. Classification confidence refers to this highest probability. Our heuristic (line 6) is that the individual with the lower classification confidence is the one who was treated unfairly. Fourth, RankByInfluence (line 10) identifies the datapoints in the original (biased) training dataset D responsible for discrimination against these individuals. Given a trained model and a set of datapoints along with their predictions from the model, RankByInfluence ranks the training data of the model from most influential to least influential training datapoint responsible for those predictions. If the most influential datapoints are removed from the training data, and the model is retrained with the same model architecture, the probability of change in the prediction of the discriminatory pairs is highest. We hypothesize that RankByInfluence returns the training datapoints sorted in order of most to least bias, and our experiments support this. We show all the four steps in the left portion of Figure 1.
None of the above steps is dependent on the number of output categories, so the algorithm is applicable to multi-class classifiers. The notion of similar predictions would need to be adjusted accordingly.
Algorithm 2 first calls SortDataset to rank the datapoints in decreasing order of bias (line 3). It then iteratively removes a chunk of the most biased datapoints from the sorted original training dataset (line 6), retrains the same model architecture on the remaining datapoints, and estimates the individual discrimination of the retrained model (line 7). When the remaining discrimination in a retrained model reaches a local minimum, it returns a debiased dataset by dropping the most biased datapoints from the original training data (line 9). The size of each chunk can be adjusted as desired (it is 1/100 of the original training data in algorithm 2).

EVALUATION
We conducted experiments to answer the following research questions: We compared our pre-processing technique against 7 other techniques: a baseline model trained on the full training dataset (Full), five pre-processing techniques -simple removal of the sensitive attribute (SR), Disparate Impact Removal (DIR) [36] (used at the highest fairness enforcing level, repair=1), Preferential Sampling (PS) [58], Massaging (MA) [58], and Learning Fair Representations (LFR) [123] -and one in-processing technique -Adversarial Debiasing (AD) [124]. The implementations for some of these techniques were taken from IBM AIF360 [8].
We evaluated the techniques using six real-world datasets that are commonly used in the fairness literature (see table 2).
For all experiments, the machine learning model architecture we used was a neural network with 2 hidden layers. We trained models with 240 different hyperparameter settings: • In the first hidden layer, the number of neurons is 16, 24, or 32.
• In the second hidden layer, the number of neurons is 8 or 12.

Experimental methodology
To create the models for our approach (to answer RQ1 and RQ3), we executed the following methodology: (1) For each of the 240 choices of hyperparameters: (a) Split the dataset into the first 80% training and last 20% testing (without randomness, but depending on the data permutation, which is one of the hyperparameters). The dataset is normalized before usage. (b) Debias the training dataset using Algorithm 2; that is, remove some points from it. (c) Compute a "debiased model", which is trained on the debiased training dataset. (2) Let the "unfair datapoints" for a dataset be the union of the datapoints removed by all the 240 models: that is, any datapoint removed by any debiasing step. To create the models for other approaches (to answer RQ2 and RQ4), we ran each approach 240 times, once for each choice of hyperparameters. The in-processing technique AD [124] does not take hyperparameters other than data permutations, therefore we only repeated the process 20 times, once for each data permutation.
To measure the performance of the models, we executed the following methodology for each trained model: (1) Measure the model's individual discrimination using the function DiscmTest of Algorithm 2 (RQ1 and RQ2). This experimental methodology addresses an observation made by Wick et al. [113] that in most previous discrimination mitigation approaches, there is a discrepancy between the algorithm and its evaluation. Previous authors have agreed on the existence of bias in the data and have devised algorithms to mitigate bias in the resulting data or classifier, but they evaluated the accuracy of their approach on the original test set, which is potentially biased. Due to this discrepancy, most previous work suggests that a technique must trade off fairness and accuracy [9,15,16,20,21,36,37,58,61,66,83,121], which Wick et al. [113] refute.
Using our experimental methodology, the classifier debiased using our approach usually has higher accuracy on the debiased test set than the classifier trained on the full dataset (see sections 4.4 and 4.5). We think of this phenomenon as a classifier with improved generalization, an intuition also shared by Berk et al. [9], who remark that fairness constraints might act as regularizers and improve generalization.
Another advantage is that using the same test set for a particular set of hyperparameters provides an apples-to-apples comparison of our technique with all the seven baselines. For the Full baseline, the full training set is used, while the test set is still debiased.

Experiments with input space threshold =0
In the first set of experiments we used the following distance functions with the definitions of individual discrimination.
Input space similarity condition: We consider two individuals to be similar if they are the same in all non-sensitive features.
Output space similarity condition: We consider two outcomes to be similar if they are the same outcome. Note that for all our experiments, the outcomes are binary.
Generating similar individuals: For the given similarity condition in the input space, GenerateSimilarPairs randomly generates the first individual, and then flips its sensitive feature to generate an individual similar to it (similar procedure as used for the hypothetical dataset in Section 2). For example, the Salary dataset has features sex, rank, age, degree, and experience, out of which age When generating individuals, there is no guarantee that the generated individuals are representative of the actual population. For example, consider a dataset whose features include gender and college alma mater. It might generate a datapoint for a male who graduated from a women's college. Such individuals are uncommon (Timothy Boatwright of Wellesley College is one example). Therefore, a dataset with many such individuals might or might not be useful in determining whether there is discrimination against graduates of the women's college. As another example, consider determining whether a basketball coach has discriminated ethnically in selecting team members; generated individuals might not be representative since Bolivian men have an average height of 160cm (5'1"), and Bosnian men have an average height of 184cm (6'). Data selection (as opposed to generation) approaches can guarantee that the individuals are characteristic, but they require accurate characterizations of, or large samples from, the population (which we don't have access to). Even so, it may not be possible or easy to find many similar pairs of individuals to compare. We acknowledge these limitations. Future work should explore how to obtain similar pairs that are characteristic of real-world populations.

Experiments with input space threshold >0
The second set of experiments used these distance functions.
Input space similarity condition: We consider two individuals to be similar if, among the non-sensitive features, they have the same value for all categorical features and are within a 10% range for all numerical features (after normalization).
Output space similarity condition: We consider two outcomes similar if they are the same outcome.
Generating similar individuals: GenerateSimilarPairs randomly generates the first individual A1. It then generates 2 similar individuals for A1 following the input space similarity condition. (Therefore, in this set of experiments, the size of the pool of similar pairs was equal to 200 times the size of the dataset.) A similar individual is generated by maintaining the same values for all categorical features, and random sampling within the −10% to +10% range for all numerical features.

Results for input space threshold = 0
The left portion of table 3 reports, for each technique, the best (lowest) individual discrimination among its 240 models. (Figure 2 plots, for each technique, the individual discrimination of all 240 models.) When multiple models have the lowest discrimination, we choose the model with the highest accuracy. Answer to RQ1: For all datasets, our technique achieves 0% remaining discrimination. Answer to RQ2: SR also achieves 0% individual discrimination. This follows from our choice of input space similarity condition. When the sensitive feature is removed, the remaining features are the same for all pairs of similar individuals. And therefore, SR gives the same prediction for two individuals with the same features. LFR and PS leave little remaining discrimination. The other techniques (DIR, MA, and AD) are not effective in eliminating individual discrimination.
The center portion of table 3 reports, for each technique, the accuracy of the lowest-discrimination model. Answer to RQ3: Our technique produces models with better accuracy than models trained on the entire training dataset. This agrees with the observation made by Wick et al. [113]: "it requires no stretch of credulity to imagine that various personal attributes (e.g., race, gender, religion; sometimes termed 'protected attributes') have no bearing on a person's intelligence, capability, potential, qualifications, etc., and consequently no bearing on the ground truth classification labels -such as job qualification status -that might be functions of these qualities. It then follows that enforcing fairness across these attributes should on average increase accuracy. " Answer to RQ4: DIR, PS, and LFR degrade the accuracy; AD and MA affect it little; and SR improves it, though not as much as our technique does. Our approach is always either best or within 3 percentage points of best, and is best on average.
A user may train multiple models (e.g., using different hyperparameter choices) and choose the best one for their application. Table 3 assumes that the user chooses the least-discriminating model. Table 4, by contrast, assumes that the user chooses the most accurate model. And table 5 assumes that the user chooses the model with the least statistical disparity. These are the extremal points on the Pareto frontier; a user might also choose a point between them [48].
The left and center portions of table 4 report, for each technique, the individual discrimination and accuracy of the most accurate model among the 240 models. We can answer RQ1-RQ4 about these models. Answer to RQ1 and RQ2: Our approach achieves, on average, nearly 0% (0.0016%) remaining discrimination; under      the definition of our input space similarity condition, SR achieves 0% discrimination; other techniques are much higher. Answer to RQ3 and RQ4: Our approach achieves by far the highest average accuracy: 93% compared to the Full model's 84% accuracy and MA's 87% accuracy. We also measured the statistical disparity: the absolute difference between the success rate for individuals with one value for the sensitive attribute and the success rate for individuals with the other value for the sensitive attribute. The right portions of tables 3 to 5 report the results. Answer to RQ5: Our technique achieves a lower statistical disparity than the baseline model trained on the full dataset. Compared to the other techniques (which are designed to optimize for statistical parity difference), our technique is in the middle of the pack. For each technique that has a lower statistical disparity than ours, our technique achieves considerably lower individual discrimination and higher accuracy.
If a bias mitigation technique is highly sensitive to hyperparameter choices, then users might need to run it many times to achieve      desirable performance, and they may have less confidence in its generalizability. Figure 2 shows the remaining individual discrimination for all 240 models we trained for each experiment and each technique, and fig. 3 shows the test accuracy for all the 240 models for each experiment and each technique.
Answer to RQ6: Our technique not only usually results in lower remaining discrimination and higher accuracy than previous techniques, it is also much less sensitive to hyperparameter choices (narrower range).

Results for input space threshold > 0
Tables 6 to 8 report the same statistics for the experiment with > 0. We answer RQ1-RQ4 using Table 6 and Table 7. Answer to RQ1: For both the models with the least discrimination and highest accuracy, our approach achieves very low individual discrimination (0.65% and 0.73% respectively).
Answer to RQ2: For the most accurate model, our approach gets the lowest discrimination among all approaches (0.73%). For the models with the least discrimination, our approach is second  Figure 2: The individual discrimination for all 240 hyperparameter choices (lower is better). Our approach (rightmost in each boxplot) always achieves 0% discrimination for some (often many) hyperparameter choices, and it has a little variance across choices.
and very close to the best performing approach (LFR). Notably, SR has much higher discrimination for this choice of similarity metric. In the previous set of experiments, SR was able to get 0% individual discrimination just by choice of input space similarity.
Answer to RQ3: Our approach gets the best accuracy by far compared to all the baselines for both the models with the least discrimination and highest accuracy.
Answer to RQ4: Similar to the conclusions from the experiments with = 0, for the least discriminative model: DIR, PS, and LFR degrade the accuracy; AD, MA, and SR affect it little. For the most accurate model: SR, MA, and LFR improve the accuracy but not as much as our approach.
Answer to RQ5: For statistical disparity (shown in the right portions of tables 6 to 8) the conclusions are the same as when = 0. Answer to RQ6: Due to lack of space, we have added the plots showing the remaining discrimination and test accuracy for the experiments with > 0 in the appendix (Appendix A). The conclusions are the same as when = 0.

RELATED WORK 5.1 Fairness Metrics
More than 20 metrics for fairness have been proposed [110]. They can be broadly divided into three categories: group fairness, causal fairness, and individual fairness.
Group fairness. Most group fairness metrics declare a model to be fair if it is satisfies a specific constraint from the confusion matrix [106] e.g., predictive parity [20,101] (equal probability of being correctly classified into the favorable class for all demographic groups), predictive equality [20,21] (equal true negative rate for all groups), equality of opportunity [20,47,70] (equal true positive rate for all groups), equalized odds [47,121] (equal true positive rate and equal true negative rate for all groups), accuracy equality [101] (equal predictive accuracy for all groups), treatment equality [101] (equal ratio of false negatives and false positives for all groups), calibration [20,47], well-calibration [66], and balance for positive and negative classes [66]. Most definitions in the group fairness category require the ground truth, which is unavailable before deploying a model [72].
The most popular definition from the group fairness category is statistical parity, which states that a decision-producing system  Figure 3: The test accuracy for all 240 hyperparameter choices (higher is better). Our approach is best or comparable to the best in terms of both accuracy and its variance, for all experiments except D7.
is fair if the probability of getting a favorable outcome is equal for members of all demographic groups formed by sensitive features.
Most discrimination-reducing approaches evaluate their methodology on statistical parity or measures close to it [15,16,36,38,58,61,112,123]. Dwork et al. [33] criticize statistical parity by showing how three evils (reduced utility, self-fulfilling prophecy, and subset targeting) can occur even while statistical parity is maintained. Statistical parity is also not applicable in scenarios where the base rates of ground-truth occurrence are different, e.g., criminal justice [20,33,47,66].
Causal fairness. Causal fairness metrics require a causal model that is used to reason about the effects of certain features on other features and the outcome [79,96]. Causal models are represented by a graph with features as nodes and directed edges showing the effects of one feature on another. Learning causal models from the data is not always possible and therefore requires domain knowledge Pearl [87]. A causal model is consequently an untestable assumption about the data generation process. Papers proposing causal fairness definitions assume a given causal model and evaluate their methodology based on this assumption [65,70,85].
Individual fairness. Individual fairness [33,54,71,100] (defined in section 1) states that similar individuals should be treated similarly: they should be given similar predictions. The similarity metric for individuals should only consider the non-sensitive features as that ensures adherence to anti-discrimination laws. This matches a common intuition, does not make assumptions about data generation [49] or base rates, and does not require the presence of ground truth labels.
Trading off group and individual fairness. Both group fairness and individual fairness are desirable. It is a policy and political decision which one to prioritize. (A related policy/political question is what forms of affirmative action, if any, are just.) Previous work has largely ignored individual fairness, which we argue is an oversight.
Our experiments show that group and individual fairness must be traded off in a relative sense: maximizing one leads to the other taking on a non-maximal value. However, they do not need to be traded off in an absolute sense: while maximizing one, it is still possible to improve the other. Our technique is a bright spot: in tables 3 to 5, ours is the only technique (out of eight) that always improves test accuracy, individual discrimination, and statistical parity. Its interventions may be acceptable across the political spectrum. This is an exciting new direction for research in fairness in machine learning.

Fairness Literature
Most previous work in the fairness literature [32,38,82,114] can be categorized into discrimination detection and interventional approaches.
Discrimination detection [3,11,39,107] measures whether a learned model is biased. Our experiments take inspiration from Themis [39], which measures individual discrimination by generating similar individuals, who only differ in a sensitive attribute, sampling individuals uniformly at random from the feature space, which is captured from training data.
Interventional approaches aim to improve the fairness of a learned model. They can be further categorized into three groups based on the stage of machine learning pipeline they intervene in: (1) Pre-processing: Intervention at the stage of training data. Previous work modifies the training data to reduce discrimination and measures it using their preferred fairness metric [14, 16, 35, 36, 43-45, 51, 56-58, 64, 73, 75-77, 81, 84, 98, 105]. Merely removing the sensitive feature (the technique called SR in this paper) does not necessarily yield a fair model [33,78,92,109,111,117] and is important to assess disparities [13,55,74]. A model can learn to make decisions based on proxy features that are correlated with sensitive attributes, e.g., zip code can encode racial groups. Removing all the proxies would cause a large dip in accuracy [117]. (2) In-processing: Intervention at the stage of training. Previous work modifies the learning algorithm [15,34,59,97], modifies the loss function [7,61,62,123], uses an adversarial approach [10,112,124], or adds fairness constraints [2,19,27,37,95,121,122]. (3) Post-processing: Intervention at the stage of deployment of a trained model. Previous work modifies the input before passing it to the model [1,18,47,70,90,119], or modifies the model's prediction depending on the input's sensitive attribute [59,60,83,88,89]. Since the sensitive attribute is used to affect model predictions directly, this class of approach might be illegal due to disparate treatment laws [52,118].

CONCLUSION
Building fair machine learning models is required for adherence to anti-discrimination laws; it leads to more desirable outcomes (e.g., higher profits); and it is the morally right thing to do. Training a machine learning model on biased historical decisions would perpetuate injustice. We have proposed a novel approach to improve fairness. Our approach heuristically identifies unfair decisions made by a model, uses influence functions to identify the training data (e.g., biased historical decisions) that are most responsible for the unfair decisions, and then removes the biased training points.
Compared to a baseline model that is trained on historical data without removing any datapoints, our technique improves test accuracy, individual discrimination, and statistical disparity. Ours is the only technique (out of eight tested) that improves all three measures, no matter which is chosen as the optimization goal. By contrast, much previous work increases fairness only at the expense of accuracy.  Figure 5: The test accuracy for all 240 hyperparameter choices (higher is better). Our approach is best or comparable to the best in terms of both accuracy and its variance, for all experiments except D2 and D7.