Predictive models are increasingly deployed for the purpose of determining access to services such as credit, insurance, and employment. Despite potential gains in productivity and efficiency, several potential problems have yet to be addressed, particularly the potential for unintentional discrimination. We present an iterative procedure, based on orthogonal projection of input attributes, for enabling interpretability of black-box predictive models. Through our iterative procedure, one can quantify the relative dependence of a black-box model on its input attributes.The relative significance of the inputs to a predictive model can then be used to assess the fairness (or discriminatory extent) of such a model.
Most of the decisions in the today's knowledge society are taken on the basis of historical data by extracting models, patterns, profiles, and rules of human behavior in support of (automated) decision making. There is then the need of developing models, methods and technologies for modelling the processes of discrimination analysis in order to discover and prevent discrimination phenomena. In this respect, discrimination analysis from data should build over the large body of existing legal and economic studies. This paper intends to provide a multi-disciplinary survey of the literature on discrimination data analysis, including methods for data collection, empirical studies, controlled experiments, statistical evidence, and their legal requirements and grounds. We cover the following mainstream research lines: labour economic models, (quasi-)experimental approaches such as auditing and controlled experiments, profiling-based approaches such as racial profiling and credit markets, and the recently blooming research on knowledge discovery approaches. In the context of civil rights law, discrimination refers to unfair or unequal treatment of people based on membership to a category or a minority, without regard to individual merit. Rules extracted from databases by data mining techniques, such as classification or association rules, when used for decision tasks such as benefit or credit approval, can be discriminatory in the above sense. In this paper, the notion of discriminatory classification rules is introduced and studied. Providing a guarantee of non-discrimination is shown to be a non trivial task. A naive approach, like taking away all discriminatory attributes, is shown to be not enough when other background knowledge is available. Our approach leads to a precise formulation of the redlining problem along with a formal result relating discriminatory rules with apparently safe ones by means of background knowledge. An empirical assessment of the results on the German credit dataset is also provided.