To do this, we leverage the ProPublica COMPAS dataset, which includes data about more than \(7,000\)  defendants arrested in Broward County Florida [1]}, [2]}. The dataset includes a recidivism risk score, computed by the COMPAS algorithm (which is meant to reflect the risk of a defendant committing a misdemeanor or felony within 2 years of assessment based on a set of features about the defendant, not including raceMore precisely, the COMPAS algorithm makes its predictions from 137 features about a defendant and the defendant’s past criminal record. COMPAS does not consider the defendant's race; however, other features it does consider may be correlated with race and thus lead to racially disparate predictions.), along with the actual outcome of whether each defendant re-offended. ProPublica found that these risk scores are inaccurate and racially biased [1]}. Further research found that human subjects with limited to no criminal justice experience exhibit similar inaccuracies and racial biases as COMPAS when predicting recidivism based on a simple prompt describing a defendant [4]}. The human subject experiment examined two conditions, one in which a defendant's race was excluded from the prompt, and one in which it was included.Interestingly, the researchers found that the exclusion of race had no significant impact on human recidivism prediction accuracy or fairness [4]}. Here, we use the same prompts outlined in [4]} but instead ask language models [7]} instead of people to predict recidivism. We leave full technical details and (significant) caveats in Appendix REF ; however, we foreground here that benchmark risk assessment instrument datasets like COMPAS often contain numerous measurement biases and errors which can make them ill-suited for making claims about real-world impact without carefully considering the the complicated socio-technical systems (in this case, the US criminal justice system) in which they are used [2]}.
<FIGURE>