<jupyter_start><jupyter_text>COMPAS Case Study: Fairness of a Machine Learning This notebook explores the difficulties in making algorithms fair to all groups. In this example, we explore the performance of COMPAS - an algorithm used by parole boards in prisons in the US to assess risk of re-offending ('recidivism').The notebook is reconstructed largely from <NAME>'s work at https://towardsdatascience.com/compas-case-study-fairness-of-a-machine-learning-model-f0f804108751The above analysis was based on orginal wotk by ProPublica: https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencingAdditional material included is from *The Alignment Problem* by <NAME>. The use of evidence in parole decisions*"Our law punishes people for what they do, not who they are. Dispensing punishment on the basis of an immutable characteristic flatly contravenes this guiding principle"*, US Supreme Court Chief Justice <NAME>.In 1927, the new chairman of the Parole Board of Illinois, <NAME>, commissioned a study on the workings of the parole system in the state. He was particularly interested in whether the right people for parole could be selected - those who would not commit further crime. On reviewing evidence, he concluded that certain characteristics, such as having served less than a year, or coming from a farming background, were associated with a lower probability of re-offending. Meanwhile other characteristics, such as those "living in the criminal underworld" were associated with a higher probability of re-offending. He recommended that the Parole Board be given a summary sheet for each man to be paroled which would list key attributes of the person and their association with re-offense rates. By 1951 a *'Manual of Parole Prediction'* had been written for Illinois, outlining the evidence gathered. At the end of the book is a present section on *'Scoring by Machine Methods'*, which considers use of punch-card machines to automate processing of data and outputting individualized prediction of likelihood to re-offend. Update of such methods was, however, slow. By 1970 only two states in the US used predictive modeling in parole decisions.But that was to change with the development, in 1998, of *'Correctional Offender Management Profiling for Alternative Sanctions'*, or *'COMPAS'*. About COMPASCOMPAS, an acronym for *Correctional Offender Management Profiling for Alternative Sanctions*, is an assistive software and support tool used to predict recidivism risk, that is the risk that a criminal defendant will re-offend.  * COMPAS is helpful in ways that it provides scores from 1 (being lowest risk) to 10 (being highest risk).* It also provides a category based evaluation labeled as high risk of recidivism, medium risk of recidivism, or low risk of recidivism. For simplifying things we can convert this multi-class classification problem into binary classification combining the medium risk and high risk of recidivism vs. low risk of recidivism.* The input used for prediction of recidivism is wide-scale and uses 137 factors including age, gender, and criminal history of the defendant as the input. COMPAS **does not** use race as an input.* Race is not an explicit feature considered by the model. Use of COMPAS, and the bias uncovered by ProPublicaCOMPAS has been used by the U.S. states of New York, Wisconsin, California, Florida’s Broward County, and other jurisdictions. Depending on the scores generated by this software, the judge can decide upon whether to detain the defendant prior to trial and/or when sentencing. It has been observed that the Defendants who are classified medium or high risk (scores of 5–10), are more likely to be held in prison while awaiting trial than those classified as low risk (scores of 1–4). Although this software might seem to be assistive and helpful but it suffers from machine bias. According to an investigative journal ProPublica:* Though the accuracy is similar for white and black people, the prediction fails differently for the black defendants:|                                           | White | African-American ||-------------------------------------------|-------|------------------|| Labelled high risk, but did not re-offend | 24%   | 45%              || Labelled lower risk, but did re-offend    | 48%   | 28%              |* Overall, COMPAS correctly predicts recidivism 61 percent of the time. But black people are almost twice as likely as white people to be labeled a higher risk but not actually re-offend. It makes the opposite mistake among white people: They are much more likely than black people to be labeled lower risk but go on to commit other crimes. This a major risk of machine learning models might possess and when it comes to someone’s freedom it’s a flaw that shouldn’t go unnoticed.To get what ProPublica claimed let’s try to make our own model which replicates their analysis.  Formulating ProPublica analysisBefore starting any analysis, we need to have the proper tools for that — importing the required libraries.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.metrics import confusion_matrix<jupyter_output><empty_output><jupyter_text>Read in the dataAfter importing the libraries we need to import the data and to know how the data looks like use the following code to visualize it.Data may be read from local file, or from the internet.<jupyter_code>read_local = False

if read_local:
    df = pd.read_csv('compas_data.csv')

else:
    url = 'https://raw.githubusercontent.com/propublica/compas-analysis/master/compas-scores-two-years.csv'
    df = pd.read_csv(url)
    df.to_csv('compas_data.csv')

df.head()<jupyter_output><empty_output><jupyter_text>As stated earlier to make this into a binary classification problem we carry out the following section.  Transform into a binary classification problemFirst, let’s make this a binary classification problem. We will add a new column that translates the risk score (decile_score) into a binary label.Any score 5 or higher (Medium or High risk) means that a defendant is treated as a likely recividist, and a score of 4 or lower (Low risk) means that a defendant is treated as unlikely to re-offend. As we know that creating a new feature with the help of the existing one might help us to envision data that might go unexplored. The code to do the same is given below:<jupyter_code># turn into a binary classification problem
# create feature is_med_or_high_risk
df['is_med_or_high_risk']  = (df['decile_score']>=5).astype(int)
df.head()<jupyter_output><empty_output><jupyter_text>Evaluate model performanceTo evaluate the performance of the model, we will compare the model’s predictions to the “truth”:* The risk score prediction of the COMPAS system is in the `decile_score` column,* The classification of COMPAS as medium/high risk or low risk is in the `is_med_or_high_risk` column* The “true” recidivism value (whether or not the defendant committed another crime in the next two years) is in the `two_year_recid` column.  AccuracyLet’s start by computing the accuracy, and the overall recidivism rate.<jupyter_code># classification accuracy
accuracy = np.mean(df['is_med_or_high_risk']==df['two_year_recid'])
print (f'Accuracy: {accuracy:0.3f}')
# Two year recidivism rate
recidivism = np.mean(df['two_year_recid'])
print (f'Recidivism: {recidivism:0.3f}')<jupyter_output>Recidivism: 0.451<jupyter_text>This, itself, might already be considered problematic…The accuracy score includes both kinds of errors:* false positives (defendant is predicted as medium/high risk but does not re-offend)* false negatives (defendant is predicted as low risk, but does re-offend)but these errors have different costs. It can be useful to pull them out separately, to see the rate of different types of errors.  Confusion matrices, false positives, and false negativesIf we create a confusion matrix, we can use it to derive a whole set of classifier metrics:* True Positive Rate (TPR) also called recall or sensitivity* True Negative Rate (TNR) also called specificity* Positive Predictive Value (PPV) also called precision* Negative Predictive Value (NPV)* False Positive Rate (FPR)* False Discovery Rate (FDR)* False Negative Rate (FNR)* False Omission Rate (FOR) Lets plot a confusion matrix of predicted vs. actual:<jupyter_code>cm = pd.crosstab(df['is_med_or_high_risk'], df['two_year_recid'], 
                               rownames=['Predicted'], colnames=['Actual'])
p = plt.figure(figsize=(5,5));
p = sns.heatmap(cm, annot=True, fmt="d", cbar=False)<jupyter_output><empty_output><jupyter_text>We can also use `sklearn`'s `confusion_matrix` to pull out these values and compute any metrics of interest:<jupyter_code>[[tn , fp],[fn , tp]]  = confusion_matrix(df['two_year_recid'], df['is_med_or_high_risk'])
print("True negatives:  ", tn)
print("False positives: ", fp)
print("False negatives: ", fn)
print("True positives:  ", tp)<jupyter_output>True negatives:   2681
False positives:  1282
False negatives:  1216
True positives:   2035<jupyter_text>We can replot our confusion matrices, normalising by row or column.<jupyter_code># Normlaise by row
cm = pd.crosstab(df['is_med_or_high_risk'], df['two_year_recid'], 
                               rownames=['Predicted'], colnames=['Actual'], normalize='index')
p = plt.figure(figsize=(5,5));
p = sns.heatmap(cm, annot=True, fmt=".2f", cbar=False)<jupyter_output><empty_output><jupyter_text>When we normalise by row we can more readily see what happened to our 'predicted' groups. We can see that for those predicted not to re-offend, 69% did not re-offend, and 31% did. For those predicted to re-offend, 61% did re-offend, and 39% did not.<jupyter_code># Normalise by column
cm = pd.crosstab(df['is_med_or_high_risk'], df['two_year_recid'], 
                               rownames=['Predicted'], colnames=['Actual'], normalize='columns')
p = plt.figure(figsize=(5,5));
p = sns.heatmap(cm, annot=True, fmt=".2f", cbar=False)<jupyter_output><empty_output><jupyter_text>When we normalise by column we can more readily see what predictions were made for the groups that did or did not re-offend. We can see that of those that did not re-offend, 68% were predicted not to re-offend, whereas 32% were predicted to offend. Of those who went on to re-offend, 63% were predicted to re-offend, whereas 37% were predicted not to re-offend. If we show false positive rate and false negative rate, we see that a defendant has a similar likelihood of being wrongly labeled a likely recidivist and of being wrongly labeled as unlikely to re-offend.<jupyter_code>fpr = fp/(fp+tn)
fnr  = fn/(fn+tp)

print(f'False positive rate (overall): {fpr:0.3f}')
print(f'False negative rate (overall): {fnr:0.3f}')<jupyter_output>False positive rate (overall): 0.323
False negative rate (overall): 0.374<jupyter_text>Analyse risk scoreWe can also directly evaluate the risk score, instead of just the labels. The risk score is meant to indicate the probability that a defendant will re-offend.<jupyter_code>d = df.groupby('decile_score').agg({'two_year_recid': 'mean'})
# plot
sns.lineplot(data=d);
plt.ylim(0,1);
plt.ylabel('Recidivism rate');<jupyter_output><empty_output><jupyter_text>Defendants with a higher COMPAS score indeed had higher rates of recidivism.  Receiver-Operator Characteristic curveWe can plot a receiver-operator characteristic curve, and calculate area under curve.<jupyter_code>from sklearn.metrics import roc_curve, roc_auc_score

fpr, tpr, thresholds = roc_curve(df['two_year_recid'], df['decile_score'])
sns.scatterplot(x=fpr, y=tpr, );
sns.lineplot(x=fpr, y=tpr);
plt.ylabel("TPR");
plt.xlabel("FPR");
auc = roc_auc_score(df['two_year_recid'], df['decile_score'])
print (f'ROC AUC: {auc:0.3}')<jupyter_output>ROC AUC: 0.702<jupyter_text>FairnessCOMPAS has been under scrutiny for issues related for fairness with respect to race of the defendant.Race is not an explicit input to COMPAS, but some of the questions that are used as input may have strong correlations with race. When we use inputs that correlate with race, race becomes a *redundant variable* - removing it will change little.First, we will find out how frequently each race is represented in the data (this is recorded, but not used in the COMPAS algorithm):<jupyter_code>df['race'].value_counts()<jupyter_output><empty_output><jupyter_text>We will focus specifically on African-American or Caucasian defendants, since they are the subject of the ProPublica claim.  Accuracy by race<jupyter_code># Restrict data to black and white pwoplw
df = df[df.race.isin(["African-American","Caucasian"])]
# compare accuracy
results = (df['two_year_recid']==df['is_med_or_high_risk']).astype(int).groupby(
    df['race']).mean()
results.round(3)<jupyter_output><empty_output><jupyter_text>It isn’t exactly the same, but it’s similar — within a few points. This is a type of fairness known as **overall accuracy equality**.  Confusion matrices, and false positives/negatives by race Calculate for black people<jupyter_code># Black
mask = df['race']=='African-American'
df_masked = df[mask]

[[tn , fp],[fn , tp]]  = confusion_matrix(
    df_masked['two_year_recid'], df_masked['is_med_or_high_risk'])

false_pos_as_frac_of_all = fp / (fp + tp + fn + tn)
false_neg_as_frac_of_all = fn / (fp + tp + fn + tn)
prop_errors_false_positive = fp / (fp + fn)
prop_no_crime_identified_as_high_risk = fp / (fp + tn)
prop_crime_identified_as_low_risk = fn / (fn + tp)

print("True negatives:  ", tn)
print("False positives: ", fp)
print("False negatives: ", fn)
print("True positives:  ", tp)
print(f"False positives as fraction of all: {false_pos_as_frac_of_all:0.3f}")
print(f"False negatives as fraction of all: {false_neg_as_frac_of_all:0.3f}")
print(f"Proportion of errors that are false positive: {prop_errors_false_positive:0.3f}")
print(f"Proportion of non-offenders who were identified as high risk: {prop_no_crime_identified_as_high_risk:0.3f}")
print(f"Proportion of offenders who were identified as low risk: {prop_crime_identified_as_low_risk:0.3f}")<jupyter_output>True negatives:   990
False positives:  805
False negatives:  532
True positives:   1369
False positives as fraction of all: 0.218
False negatives as fraction of all: 0.144
Proportion of errors that are false positive: 0.602
Proportion of non-offenders who were identified as high risk: 0.448
Proportion of offenders who were identified as low risk: 0.280<jupyter_text>Calculate for white people<jupyter_code># White
mask = df['race']=='Caucasian'
df_masked = df[mask]

[[tn , fp],[fn , tp]]  = confusion_matrix(
    df_masked['two_year_recid'], df_masked['is_med_or_high_risk'])

false_pos_as_frac_of_all = fp / (fp + tp + fn + tn)
false_neg_as_frac_of_all = fn / (fp + tp + fn + tn)
prop_errors_false_positive = fp / (fp + fn)
prop_no_crime_identified_as_high_risk = fp / (fp + tn)
prop_crime_identified_as_low_risk = fn / (fn + tp)

print("True negatives:  ", tn)
print("False positives: ", fp)
print("False negatives: ", fn)
print("True positives:  ", tp)
print(f"False positives as fraction of all: {false_pos_as_frac_of_all:0.3f}")
print(f"False negatives as fraction of all: {false_neg_as_frac_of_all:0.3f}")
print(f"Proportion of errors that are false positive: {prop_errors_false_positive:0.3f}")
print(f"Proportion of non-offenders who were identified as high risk: {prop_no_crime_identified_as_high_risk:0.3f}")
print(f"Proportion of offenders who were identified as low risk: {prop_crime_identified_as_low_risk:0.3f}")<jupyter_output>True negatives:   1139
False positives:  349
False negatives:  461
True positives:   505
False positives as fraction of all: 0.142
False negatives as fraction of all: 0.188
Proportion of errors that are false positive: 0.431
Proportion of non-offenders who were identified as high risk: 0.235
Proportion of offenders who were identified as low risk: 0.477<jupyter_text>Here we see a clear disparity by race:* 45% of black people who did not go on to re-offend were classed as high risk of offending, whereas the same figure for white people was 24%.* 48% of white people who went on to re-offend were classed as low risk, whereas the same figure for black people was 28%* 22% of all black people are incorrectly assessed as high risk, compared to 14% of white people.* 19% of white people are incorrectly assessed as low risk, compared to 14% of black people.* For black people, the algorithm incorrectly classifies more people as high risk than low risk. 60% of errors are false positives.* For white people, the algorithm incorrectly classifies more people as low risk than high risk. 43% of errors are false positives.  Positive predictive value by raceNext, let’s see whether a defendant who is classified as medium/high risk has the same probability of recidivism for the two groups.In other words, we will compute the PPV for each group:<jupyter_code># compute PPV
results = df[df['is_med_or_high_risk']==1]['two_year_recid'].groupby(
    df['race']).mean()
results.round(3)<jupyter_output><empty_output><jupyter_text>Again, similar (within a few points). This is a type of fairness known as **predictive parity**.  Assessment of risk score by race<jupyter_code># calibration plot
d = pd.DataFrame(
    df.groupby(['decile_score','race']).agg({'two_year_recid': 'mean'}))
d = d.reset_index()
im = sns.lineplot(
    data=d, x='decile_score', y='two_year_recid', hue='race');
im.set(ylim=(0,1));<jupyter_output><empty_output><jupyter_text>We can see that for both African-American and Caucasian defendants, for any given COMPAS score, recidivism rates are similar. This is a type of fairness known as **calibration**  Frequency of risk score by raceNext, we will look at the frequency with which defendants of each race are assigned each COMPAS score:<jupyter_code># frequency plot
g = sns.FacetGrid(df, col="race", margin_titles=True);
g.map(plt.hist, "decile_score", bins=10);<jupyter_output><empty_output><jupyter_text>We observe that Caucasian defendants in this sample are more likely to be assigned a low risk score.However, to evaluate whether this is *unfair*, we need to know the true prevalence — whether the rates of recidivism are the same in both populations, according to the data:<jupyter_code># base rates
results = df.groupby('race').agg({'two_year_recid': 'mean',  
                        'is_med_or_high_risk': 'mean', 
                        'decile_score': 'mean'})
results.round(3)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08540598867008363, 'fraction_numerical': 0.012516859994604801, 'mean_word_length': 5.14181577203446, 'pattern_counts': {'":': 0, '<': 75, '<?xml version=': 0, '>': 76, 'https://': 3, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '19', 'dir': 'jupyter-structured-clean-dedup', 'id': '615935', 'license': 'MIT', 'n_tokens_mistral': 5707, 'n_tokens_neox': 5291, 'n_words': 2268, 'path': 'compas.ipynb', 'repo_name': 'MichaelAllen1966/fairness', 'size': '18566'}