<jupyter_start><jupyter_text>Algorithmic Fairness: Considering Different DefinitionsApproximate notebook time: 2 hours   IntroductionDecision making within the United States criminal justice system relies heavily on risk assessment, which determines the potential risk that a released defendant will fail to appear in court or cause harm to the public. Judges use these assessments to decide if bail can be set or if a defendant should be detained before trial. While this is not new in the legal system, the use of risk scores determined by an algorithm are gaining prevalence and support. Proponents promote the use of risk scores to guide judges in their decision making, arguing that machine learning could lead to greater efficiency, accountability, and less biased decisions compared with human judgment ([Henry](https://theappeal.org/risk-assessment-explained/)). On the other hand, critical voices raise the concern that biases can creep into these algorithms at any point in the process, and that algorithms are often applied to the wrong situations ([Henry](https://theappeal.org/risk-assessment-explained/)). Further, they exacerbate the racism embedded deep within the criminal justice system by perpetuating inequalities found in historical data ([Henry](https://theappeal.org/risk-assessment-explained/)).In the debate about the use of risk assessment algorithms, people have used data analysis to determine the extent to which these algorithms are fair to different groups of people. In this homework, **you will explore some of the many definitions and metrics (different ways of operationalizing data to quantify those definitions) of fairness that can be applied to the risk assessment tool COMPAS**. In doing so, you will understand and provide evidence for or against the presence of bias within the algorithm. You will examine the arguments and analyses made by the company that created COMPAS and the critics of this risk assessment tool to gain a deeper understanding of the technical and societal interpretations and implications of fairness. **NOTE**: When we discuss bias in this module, we define it most generally as prejudice or an inclination in favor of one person, thing, or group compared to another. In the context of machine learning, bias is a “phenomenon that occurs when an algorithm produces results that are systemically prejudiced due to erroneous assumptions in the machine learning process” ([Rouse](https://searchenterpriseai.techtarget.com/definition/machine-learning-bias-algorithm-bias-or-AI-bias:~:text=Machine%20learning%20bias%2C%20also%20sometimes,in%20the%20machine%20learning%20process)).  Table of Contents:* [Part 0. COMPAS](part-zero)* [Part 1. ProPublica's Perspective](part-one)* [Part 2. Northpointe's Perspective](part-two)* [Part 3. Yet Another Definition of Fairness](part-three)* [Part 4. Conclusion](part-four)  SetupLet's begin by importing the packages we need.<jupyter_code>import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_curve, roc_auc_score
import matplotlib.pyplot as plt
import seaborn as sns
!pip install aif360
!pip install BlackBoxAuditing
from aif360.algorithms.preprocessing import DisparateImpactRemover
from aif360.datasets import BinaryLabelDataset<jupyter_output>Requirement already satisfied: aif360 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (0.3.0)
Requirement already satisfied: numpy>=1.16 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.18.5)
Requirement already satisfied: scipy>=1.2.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.5.0)
Requirement already satisfied: scikit-learn>=0.21 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (0.23.1)
Requirement already satisfied: matplotlib in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (3.2.2)
Requirement already satisfied: pandas>=0.24.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from aif360) (1.0.5)
Requirement already satisfied: joblib>=0.11 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.21->aif360) (0.16.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /Users/eva/opt/anaconda3/lib/python3.8/site-packages (from scikit-learn>=0.21->aif3[...]<jupyter_text>Part 0. COMPAS: Why it was created and how it exists in the court system COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a commercial tool produced by the for-profit company Northpointe (acquired by equivant) known as a recidivism risk assessment system. **Tools like COMPAS are used to predict the risk of future crimes for an individual who has entered the US criminal justice system by outputting a risk score from 1-10**. While COMPAS was initially intended to aid decisions made by probation officers on treatment and supervision of those who are incarcerated, Northpointe has since emphasized the scalability of the tool to “fit the needs of many different decision points” including pre-screening assessments, pretrial release decisions (whether or not to hold an arrested individual in jail until their trial), and post-trial next steps for the defendant ([Northpointe](http://www.northpointeinc.com/files/downloads/FAQ_Document.pdf)). These algorithms are believed by many to provide the ability to make the court system more just by removing or correcting for bias of criminal justice officials.  Question 0aExplain 3 parties that are impacted by the COMPAS tool. In what ways are they impacted? (Can you think of impacts beyond those in the courtroom for at least one of your examples?) *Student Written Answer Here*  Question 0bBased on your initial reading, what is one problem of the criminal justice system that the COMPAS tool could potentially alleviate? What is one potential problem that using the COMPAS algorithm could introduce?  *Student Written Answer Here*  Dataset SetupWe will be using the data that was obtained and used by ProPublica in their own analysis of the COMPAS tool from Broward County public records of people who were scored between 2013 and 2014 ([ProPublica](https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm)). In order to replicate ProPublica's analysis, we remove any cases where the charge was not within 30 days of the score (ProPublica did this in order to match the COMPAS score with the correct criminal case). We are left with 6172 rows in the dataset.<jupyter_code>data = pd.read_csv('compas-scores-two-years.csv')
data = data.query('days_b_screening_arrest <= 30 & days_b_screening_arrest >= -30')
data<jupyter_output><empty_output><jupyter_text>We are also able to filter out any information that was not used by ProPublica and select fields for severity of charge, number of priors, demographics, age, sex, compas scores, and whether each person was accused of a crime within two years.<jupyter_code>select_data = data[["age", "c_charge_degree", "race", "age_cat", "score_text", "sex", "priors_count", 
                    "days_b_screening_arrest", "decile_score", "is_recid", "two_year_recid", "c_jail_in", "c_jail_out"]]<jupyter_output><empty_output><jupyter_text>Question 0cExplore the dataset. What is the granularity of this dataset?  *Student Written Answer Here* ***Sensitive features*** are attributes within a dataset that are given special consideration and treatment for potential legal, social, or ethical reasons. Often, these features are recognized and protected by antidiscrimination or privacy laws. One example of a sensitive feature is age.   Question 0dIdentify 2 sensitive features in the dataset that we have not already mentioned.  *Student Written Answer Here*  Question 0e Pick one of the sensitive features you have identified. Identify at least 2 features in the dataset that are proxies for that sensitive feature.  *Student Written Answer Here*  Question 0fAs a data scientist, why is it important to give special consideration to these kinds of features?  *Student Written Answer Here*  Part 1. ProPublica’s Perspective  Who is ProPublica?ProPublica is a nonprofit organization that “produces investigative journalism with moral force” ([ProPublica](https://www.propublica.org/about/)). ProPublica was founded as a nonpartisan newsroom aiming to expose and question abuses of power, justice, and public trust, often by systems and institutions deeply ingrained in the US.In 2016, ProPublica investigated the COMPAS algorithm to assess the accuracy of and potential racial bias within the tool, as it became more popular within the United States court system nationwide. In their analysis, ProPublica used data from defendants with risk scores from Broward County, FL from 2013 to 2014 to test for statistical differences in outcomes for Black and white defendants, which ultimately highlighted racial disparities that exist within the algorithm. ProPublica came to the conclusion that COMPAS utilizes data from a criminal justice system with a history of racial injustices, thus continuing to disproportionately target and arrest Black people in comparison to their white counterparts. While the COMPAS algorithm treats unequal groups alike, which may appear neutral, ProPublica’s data analysis and reporting emphasized the bias against Black defendants and their communities that COMPAS produced from this line of thinking, a claim that Northpointe has disputed (as we will see later).Let's retrace ProPublica's statistical analysis in order to better understand ProPublica's argument and engage with the metric of fairness that it uses.  Question 1. Logistic Regression: What are the odds of getting a high risk score?ProPublica’s first attempt at understanding the disparity in risk scores from the COMPAS tool was through logistic regression to model the chance of getting a “higher” (i.e. more "risky") score. COMPAS labels scores 1-4 as low, 5-7 as medium, and 8-10 as high scores. For the purposes of their analysis, ProPublica labeled any score above a low score as high.  Question 1a (i)Create a logistic regression model to predict the score of defendants based on their sex, age, race, previous arrests, seriousness of the crime, and future criminal behavior.<jupyter_code># Create logistic regression model
# Student Code Here<jupyter_output><empty_output><jupyter_text>Question 1a (ii)Print out the coefficients paired with the corresponding feature names.<jupyter_code># Pair the coefficients with feature names
# Student Code Here<jupyter_output><empty_output><jupyter_text>Question 1bWhat features are most predictive?  *Student Written Answer Here*  Question 1cAre Black defendants more likely to get a high risk score opposed to white defendants? If so, by how much? Show your calculations.<jupyter_code># Student Calculations Here<jupyter_output><empty_output><jupyter_text>*Student Written Answer Here*   Question 2. FPR and FNR: Does COMPAS overpredict or underpredict across groups?In order to answer this question and understand the ways in which bias is present in the risk scores, ProPublica used the False Positive Rate (FPR) and False Negative Rate (FNR) as their metrics to understand and quantify fairness.   Question 2aComplete the following functions to calculate the FPR and FNR. Afterwards, apply these functions to each racial subgroup: Black defendants and white defendants. Keep in mind that ProPublica defines a high score as anything above 4, and therefore a false positive would be a defendant with a high score who did not recidivate.<jupyter_code>def fpr(race_feature, data):
    # Return the False Positive Rate of scores for the specified race_feature

    return 0


def fnr(race_feature, data):
    # Return the False Negative Rate of scores for the specified race_feature

    return 0

# Apply the metrics to the dataset - Replace zeros with correct metrics
print("FPR for Black defendants:", 0)
print("FPR for white defendants:", 0)
print("FNR for Black defendants:", 0)
print("FNR for white defendants:", 0)<jupyter_output>FPR for Black defendants: 0
FPR for white defendants: 0
FNR for Black defendants: 0
FNR for white defendants: 0<jupyter_text>Question 2bWhat can you conclude from these metrics about the overprediction of risk scores for Black and white defendants? By how much is the tool overpredicting?  (Hint: Look at your calculations for the FPR.)<jupyter_code># Calculation of ratio for FPR<jupyter_output><empty_output><jupyter_text>*Student Written Answer Here*  Question 2cWhat can you conclude from these metrics about the underprediction of risk scores for Black and white defendants? By how much is the tool underpredicting? (Hint: Look at your calculations for the FNR.)<jupyter_code># Calculation of ratio for FNR<jupyter_output><empty_output><jupyter_text>*Student Written Answer Here*  Question 2dWhat is the importance of overprediction and underprediction in regard to ProPublica’s analysis? How might these observations have real impacts on the defendants who receive scores? *Student Written Answer Here*   Question 3.  Question 3a (i)Utilizing your answers from 1b and 2b, what problems does ProPublica highlight in the COMPAS algorithm?  *Student Written Answer Here*  Question 3a (ii)How would you describe ProPublica’s definition of fairness, after learning and utilizing the metrics they used? *Student Written Answer Here*  Question 3b Why did ProPublica choose to investigate bias between races rather than a different sensitive feature? (Hint: think about how ProPublica’s conclusions reflect the racial disparities in our current criminal justice system.) *Student Written Answer Here*  Question 3cWhat is ProPublica’s agenda as an investigative journalism organization? How do we see this in their analysis and conclusions? *Student Written Answer Here* We mentioned earlier that Northpointe disagreed with ProPublica's argument that the COMPAS algorithm is racially biased. Now that we’ve analyzed ProPublica’s perspective and seen the way in which they define and operationalize the concept of fairness, let’s move on to Northpointe’s.  Part 2. Northpointe's Perspective  Who is Northpointe? Northpointe (merged with two other companies to create *equivant* in 2017) is a for-profit computer software company that aims to advance justice by informing and instilling confidence in decision makers at every stage of the criminal justice system ([equivant](https://www.equivant.com/)). In addition to operating and continuing to develop COMPAS, *equivant* has developed a variety of technologies for use in court case management, attorney case management, inmate classification, and risk/needs assessment strategies. In the wake of criticism from ProPublica and other researchers alike, Northpointe produced a [detailed response](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf) to ProPublica’s allegations, claiming that these critiques of their tool utilized the wrong type of classification statistics in their analysis and portrayed the tool incorrectly. The company provided their own analysis of the COMPAS algorithm by using different statistical methods and responding individually to each of ProPublica’s claims of racial bias against Black defendants. Upon examining their tool’s fairness through accuracy equity and predictive parity (which are metrics that were left out of ProPublica’s analysis), as well as the fact that the model was not trained with a race feature, Northpointe concluded that their algorithm treats all citizens and specified groups equally, and therefore does not exhibit signs of bias or inequality for specified groups. Now, let’s take a look at how Northpointe supported this argument.  Question 4. Accuracy Equity: Is each group being discriminated against equally?Instead of analyzing and comparing the model errors FNR and FPR, Northpointe utilized the complement of FNR, known as the TPR (or what is often referred to as *Sensitivity*), paired with the FPR to prove what they refer to as ***Accuracy Equity*** through the use of a *ROC Curve*. Accuracy equity, according to [Northpointe](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf), is exhibited in the model “if it can discriminate recidivists and nonrecidivists equally well for two different groups such as blacks and whites.” Recall that we use ROC curves and the *Area Under the Curve* to understand how much a model is capable of distinguishing between classes.   Question 4aUtilize the sklearn metrics package to calculate TPR and FPR, visualize the ROC curve for both white and Black defendants, and calculate the AUC for each curve.<jupyter_code># Calculate FPR and FNR from metrics package - Black defendants

# Calculate FPR and FNR from metrics package - White defendants

# Plot the ROC 
plt.subplots(1, figsize=(10,10))
plt.title('ROC - Black Defendents')
plt.ylabel('Sensitivity')
plt.xlabel('1 - Specificity')
plt.plot([0, 1], ls="--")
plt.show()

# Calculate the AUC - Replace zeros with correct AUC's
print("AUC for Black defendants:", 0)
print("AUC for white defendants:", 0)<jupyter_output>AUC for Black defendants: 0
AUC for white defendants: 0<jupyter_text>Question 4b (i)What do you notice from the ROC curve and the AUC calculation? List at least two general observations.  *Student Written Answer Here*  Question 4b (ii)What could Northpointe take away from this visualization to prove their point? Is accuracy equity being represented here? (Hint: Is each racial group being discriminated against equally?) *Student Written Answer Here*  Question 5. Predictive Parity: Is the likelihood of recidivism equal across groups?In addition to the metric outlined above, Northpointe also utilized positive predictive values to explore the likelihood of defendants to reoffend, and to therefore prove that ***Predictive Parity*** is achieved. Predictive parity, according to [Northpointe](http://go.volarisgroup.com/rs/430-MBX-989/images/ProPublica_Commentary_Final_070616.pdf), is exhibited in a model “if the classifier obtains similar predictive values for two different groups such as blacks and whites, for example, the probability of recidivating, given a high risk score, is similar for blacks and whites.” Let’s explore how they analyzed this.   Question 5a Complete the following functions to calculate the positive predictive values and negative predictive values. Afterwards, apply these functions to the data of white defendants and the data of Black defendants.<jupyter_code>def ppv(race_feature, data):
    # Return the Positive Predictive Value of scores for the specified race_feature
    
    return 0


def npv(race_feature, data):
    # Return the Negative Predictive Value of scores for the specified race_feature
    
    return 0

# Apply metrics to the dataset - Replace zeros with correct metrics
print("PPV for Black defendants:", 0)
print("NPV for Black defendants:", 0)
print("PPV for white defendants:", 0)
print("NPV for white defendants:", 0)<jupyter_output>PPV for Black defendants: 0
NPV for Black defendants: 0
PPV for white defendants: 0
NPV for white defendants: 0<jupyter_text>Question 5bUse the metrics you calculated above to fill in the table below. | |  White  | Black    ||---:|:-------------|:-----------|| Labeled higher risk, but didn't reoffend | % | % || Labeled lower risk, but did reoffend | % | % |<jupyter_code># Student calculations for above table<jupyter_output><empty_output><jupyter_text>Question 5c (i)What do you notice about the positive predictive values for each group? List at least one general observation. *Student Written Answer Here*  Question 5c (ii)What could Northpointe conclude from these findings? Is predictive parity represented here? (Hint: Is the likelihood of recidivism relatively equal for each racial group?) *Student Written Answer Here*  Question 6.  Question 6aHow would you describe Northpointe’s definition of fairness, after learning and utilizing the metrics they used? How is this different from your description of ProPublica’s definition from Q3aii?  *Student Written Answer Here*  Question 6b If anything, what are ProPublica and Northpointe each not considering in their definitions? (Hint: Think about other goodness metrics in ML, as well as your knowledge of the historical context of policing data) *Student Written Answer Here* So far, we’ve investigated ProPublica’s and Northpointe’s definitions of fairness. In the world of machine learning there are [many more](https://www.google.com/url?q=https://fairmlbook.org/tutorial2.html&sa=D&ust=1606727018134000&usg=AOvVaw06zU_fm8h7xp71d8igA8KI), so in the next section we will take a look at a third definition.  Part 3. Yet Another Definition of Fairness In this section, you will go through yet another metric and definition used to evaluate fairness in machine learning: **disparate impact**. Disparate impact is a legal doctrine that determines if there is unintended discrimination towards a protected class ([Society for Human Resource Management](https://www.shrm.org/resourcesandtools/tools-and-samples/hr-qa/pages/disparateimpactdisparatetreatment.aspx)). In machine learning, disparate impact is a metric to evaluate fairness in a model. It is a form of bias within an algorithm that reflects systemic discrimination when a model’s outputs are dependent on a ***sensitive feautre*** (the protected class). This is often considered unintentional (like the legal doctrine) due to the fact that the sensitive feature is omitted from the model, though it is still correlated with the output through proxy variables ([Wang et al.](https://arxiv.org/pdf/1801.05398.pdf:~:text=Abstract%E2%80%94In%20the%20context%20of,e.g.%2C%20race%20or%20gender)).Not only will you evaluate the fairness of the tool (as Northpointe and ProPublica did) by measuring the bias reflected in the outputs of the model, but you will remove it to actually change those outputs and therefore eliminate the dependencies between the risk scores and the race feature. In order to computationally remove the disparate impact that we quantify, we can use tools like aif360’s [Disparate Impact Remover](https://aif360.readthedocs.io/en/latest/modules/generated/aif360.algorithms.preprocessing.DisparateImpactRemover.html). aif360 is a package created by IBM’s AI Research team, which contains a variety of tools to “help you examine, report, and mitigate discrimination and bias in machine learning models throughout the AI application lifecycle” ([AI Fairness 360](https://aif360.mybluemix.net/)).  Question 7. Disparate Impact: Quantification and RemovalFirst, let’s visualize the disparity that we would like to remove from the dataset. In order to do that we need to distinguish between a privileged group and an unprivileged group. In technical terms, the privileged group receives higher scores from a trained model, so therefore the Black defendants will be considered "privileged" and the white defendants will be considered "unprivileged" in this case.   Question 7aUse a histogram to plot the scores for Black defendants and the scores for white defendants. Visualize both histograms on the same plot.<jupyter_code># Student Code for Histogram<jupyter_output><empty_output><jupyter_text>Question 7b What do you notice from the plot? (Hint: how do the distributions differ across racial groups with respect to mean, shape of distribution, etc.)  *Student Written Answer Here* Now, we need to quantify the disparate impact we are seeing in the plot. In machine learning, we can understand disparate impact as the proportion of individuals that get positive outcomes (did they get a high score) for the two groups described above: \begin{align}\Pr(Y=1|D=Unprivileged) \ / \ \Pr(Y=1|D=Privileged)\end{align} In this equation Y is 1 if the defendant received a high score and 0 if they received a low score.   Question 7c (i)Create a function to calculate the proportion of individuals from a specified racial group that get positive outcomes.<jupyter_code>def proportion(data, group):
    # Returns the proportion of individuals in data from the group who recidivate
    
    return 0<jupyter_output><empty_output><jupyter_text>Question 7c (ii)Use this function to calculate the disparate impact, using the equation from above.<jupyter_code># Student Code Here<jupyter_output><empty_output><jupyter_text>If the proportion of unprivileged individuals receiving positive outcomes to privileged individuals receiving positive outcomes is less than 80%, there is a disparate impact violation. In order to stop a trained model from replicating these biases in its output, we can now use aif360’s Disparate Impact Remover to remove the bias we just calculated.  Question 7dCreate a Disparate Impact Remover type object and use the function fit_transform on our data. In order to do this you will need to first create a BinaryLabelDataset from our dataset to use for fit_transform. Check out the documentation [here](https://aif360.readthedocs.io/en/latest/modules/generated/aif360.datasets.BinaryLabelDataset.htmlaif360.datasets.BinaryLabelDataset) to see how to implement this.<jupyter_code># Create new DataFrame with just the necessary columns - Only numeric values for race, decile score, and two year recid

# The BinaryLabelDataset requires decile_scores to be continuous -->
# Use this line of code noise = np.random.normal(0, 0.1, race_data.shape[0]) to add noise to your decile_score column
# Student Code Here<jupyter_output><empty_output><jupyter_text>Question 7eSimilar to part a, use a histogram to plot the scores on the modified dataset. Afterwards, use the proportion function created above to calculate the disparate impact of the transformed dataset.<jupyter_code># Transform output from DIRemover into usable DataFrame

# Plot the new overlayed histogram
# Student Calculation Here<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.05653314810527128, 'fraction_numerical': 0.014779655784518021, 'mean_word_length': 5.41460396039604, 'pattern_counts': {'":': 0, '<': 66, '<?xml version=': 0, '>': 77, 'https://': 14, 'lorem ipsum': 0, 'www.': 6, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '17', 'dir': 'jupyter-structured-clean-dedup', 'id': '497959', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 6891, 'n_tokens_neox': 6285, 'n_words': 3372, 'path': 'COMPAS/.ipynb_checkpoints/COMPAS Project - No Solutions-checkpoint.ipynb', 'repo_name': 'ds-modules/HCE-Materials', 'size': '25914'}