We found that language models exhibit similar (or worse) inaccuracies and racial biases as COMPAS. Figure REF  shows language models of increasing size compared to COMPAS in terms of two metrics mentioned in the ProPublica analysis [1]} and the subsequent human subject experiment [2]}: overall predictive accuracy, and the ratio in false positive rates for Black versus white defendants. We show results for both prompts that exclude an individual's race (blue) and include it (orange). For overall predictive accuracy, language models become increasingly accurate at predicting whether defendants will re-offend (Figure REF , Left) as they increase in size, yet they are still unreliable predictors like COMPAS. We see no significant difference in predictive accuracy when race is excluded from the prompt or included. In both conditions, the largest model, with  52B parameters, achieves 63% accuracy compared to COMPASs 66% accuracy.
