Synthetic Crowdsourcing: A Machine Learning Approach to Inconsistency in Adjudication
This paper presents a novel algorithmic approach to the problems of inconsistency and bias in legal decision making. First, we propose a new tool for reducing inconsistency: "Synthetic Crowdsourcing Models" "SCMs" built with machine learning methods. By providing judges with recommendations generated from statistical models of themselves, such models can help those judges make better and more consistent decisions. To illustrate these advantages, we build an SCM of release decisions for the California Board of Parole Hearings. Second, we describe a means to address systematic biases that are embedded in an algorithm (e.g. disparate racial treatment) We argue for making direct changes to algorithmic output based on explicit estimates of bias. Most commentators concerned with embedded biases have focused on constructing algorithms without the use of bias inducing variables. Given the complex ways that variables may correlate and interact, that approach is both practically difficult and harmful to predictive power. In contrast, our two step approach can address bias with minimal sacrifice of predictive performance.