The Boston housing dataset [1]} contains median home values in Boston, MA, that can be predicted by several covariates, including sensitive attributes, e.g., those related to race. Models that discriminate based off of such features necessitate that their operation be exposed by explanations.
We also evaluate explainers on the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) recidivism risk dataset [2]}.
The dataset was collected by ProPublica in 2016 and contains
covariates, such as criminal history and demographics, the proprietary COMPAS risk score, and recidivism data
for defendants from Broward County, Florida.
The FICO Home Equity Line of Credit (HELOC) dataset [3]}, introduced in a 2018 XAI challenge, is also used in this work. It comprises anonymous HELOC applications made by consumers requesting a credit line in the range of $5,000 and $150,000. Given the credit history and characteristics of an applicant, the task is to predict whether they will be able to repay their HELOC account within two years.
Last, we evaluate on a down-sampled version of the MNIST dataset [4]}. With the aim of reducing explainer runtime and improving comprehensibility of effect-wise results, we crop then resize each handwritten digit in the dataset
to \(12\times 10\)  and only include a subset of the 10 digits. See Appendix A for more details.
