Machine learned models exhibit bias, often because the datasets used to train them are biased. This presents a serious problem for the deployment of such technology, as the resulting models might perform poorly on populations that are minorities within the training set and ultimately present higher risks to them. We propose to use high-fidelity computer simulations to interrogate and diagnose biases within ML classifiers. We present a framework that leverages Bayesian parameter search to efficiently characterize the high dimensional feature space and more quickly identify weakness in performance. We apply our approach to an example domain, face detection, and show that it can be used to help identify demographic biases in commercial face application programming interfaces (APIs). Machine learned classifiers are becoming increasingly prevalent and important. Many systems contain components that leverage trained models for detecting or classifying patterns in data. Whether decisions are made entirely, or partially based on the output of these models, and regardless of the number of other components in the system, it is vital that their characteristics are well understood. However, the reality is that with many complex systems, such as deep neural networks, many of the "unknowns" are unknown and need to identified BID23 . Imagine a model being deployed in law enforcement for facial recognition, such a system could encounter almost infinite scenarios; which of these scenarios will the classifier have a blind-spot for? We propose an approach for helping diagnose biases within such a system more efficiently.Many learned models exhibit bias as training datasets are limited in size and diversity BID34 BID33 , or they reflect inherent human-biases BID7 . It is difficult for researchers to collect vast datasets that feature equal representations of every key property. Collecting large corpora of training examples requires time, is often costly and is logistically challenging. Let us take facial analysis as an exemplar problem for computer vision systems. There are numerous companies that provide services of face detection and tracking, face recognition, facial attribute detection, and facial expression/action unit recognition (e.g., Microsoft (msf, 2018) , Google (goo, 2018) , Affectiva (McDuff et al., 2016; aff, 2018) ). However, studies have revealed systematic biases in results of these systems BID6 BID5 , with the error rate up to seven times larger on women than men. Such biases in performance are very problematic when deploying these algorithms in the real-world. Other studies have found that face recognition systems misidentify [color, gender (women) , and age (younger)] at higher error rates BID22 . Reduced performance of a classifier on minority groups can lead to both greater numbers of false positives (in a law enforcement domain this would lead to more frequent targeting) or greater numbers of false negatives (in a medical domain this would lead to missed diagnoses).Taking face detection as a specific example of a task that all the services mentioned above rely upon, demographic and environmental factors (e.g., gender, skin type, ethnicity, illumination) all influence the appearance of the face. Say we collected a large dataset of positive and negative examples of faces within images. Regardless of how large the dataset is, these examples may not be evenly distributed across each demographic group. This might mean that the resulting classifier performs much less accurately on African-American people, because the training data featured few examples. A longitudinal study of police departments revealed that African-American individuals were more likely to be subject to face recognition searches than others BID15 . To further complicate matters, even if one were to collect a dataset that balances the number of people with different skin types, it is highly unlikely that these examples would have similar characteristics across all other dimensions, such as lighting, position, pose, etc. Therefore, even the best efforts to collect balanced datasets are still likely to be flawed. The challenge then is to find a way of successfully characterizing the performance of the resulting classifier across all these dimensions.The concept of fairness through awareness was presented by BID9 , the principle being that in order to combat bias we need to be aware of the biases and why they occur. This idea has partly inspired proposals of standards for characterizing training datasets that inform consumers of their properties BID20 . Such standards would be very valuable. However, while transparency is very important, it will not solve the fundamental problem of how to address the biases caused by poor representation. Nor will it help identify biases that might still occur even with models trained using carefully curated datasets.Attempts have been made to improve facial attribute detection by including gender and racial diversity. In one example, by BID29 , results were improved by scraping images from the web and learning facial representations from a held-out dataset with a uniform distribution across race and gender intersections. However, a drawback of this approach is that even images available from vast sources, such as Internet image search, may not be evenly balanced across all attributes and properties and the data collection and cleaning is still very time consuming.To address the problem of diagnosing bias in real world datasets we propose the use of high-fidelity simulations BID30 to interrogate models. Simulations allow for large volumes of diverse training examples to be generated and different parameter combinations to be systematically tested, something that is challenging with "found" data scrapped from the web or even curated datasets.Simulated data can be created in different ways. Generative adversarial networks (GANs ) BID17 are becoming increasingly popular for synthesizing data BID31 . For example, GANs could be used to synthesize images of faces at different ages BID40 . However, GANs are inherently statistical models and are likely to contain some of the biases that the data used to train them contain. A GAN model trained with only a few examples of faces with darker skin tones will likely fail to produce a diverse set of high quality synthesized images with this attribute. Parameterized graphics models are an alternative for training and testing vision models BID36 BID15 BID35 . Specifically, it has been proposed that graphics models be used for performance evaluation BID19 . As an example, this approach has been used for models for pedestrian detection BID35 . To the best of our knowledge graphics models have not been employed for detecting demographic biases within vision models. We believe that demographic biases in machine learned systems is significant enough a problem to warrant further attention.The contributions of this paper are to: (1) present a simulated model for generating synthetic facial data, (2) show how simulated data can be used to identify the limitations of existing face detection algorithms, and (3) to present a sample efficient approach that reduces the number of simulations required. The simulated model used in this paper is made available . We present an approach that leverages highly-realistic computer simulations to interrogate and diagnose biases within ML classifiers. We propose the use of simulated data and Bayesian optimization to intelligently search the parameter space. We have shown that it is possible to identify limits in commercial face detection systems using synthetic data. We highlight bias in these existing classifiers which indicates they perform poorly on darker skin types and on older skin texture appearances. Our approach is easily extensible, given the amount of parameters (e.g., facial expressions and actions, lighting direction and intensity, number of faces, occlusions, head pose, age, gender, skin type) that can be systematically varied with simulations.We used one base facial model for our experimentation. This limits the generalization of our conclusions and the ability for us to determine whether the effects would be similar or different across genders and other demographic variables. Synthetic faces with alternate bone structures would need to be created to test these hypotheses. While the initial cost of creating the models is high, they can be used to generate large volumes of data, making synthetics cost effective in the long-run. Age modeling in face images should be improved using GAN or improved parametric synthetic models. A limitation of our work is that the aging was only represented via texture changes. We plan to investigate GAN-based approaches for synthesis and compare these to parametric synthesis. A hybrid of parametric and statistical models could be used to create a more controllable but diverse set of synthesized faces. Future work will consider retraining the models using synthetic data in order to examine whether this can be used to combat model bias.
We present a framework that leverages high-fidelity computer simulations to interrogate and diagnose biases within ML classifiers. 
{"src": "scitldr-AIC_train", "paper_id": "BJf_YjCqYX"}