<jupyter_start><jupyter_text>Simpson's ParadoxUse `admission_data.csv` for this exercise.No ejecutar ninguna celda<jupyter_code>import pandas as pd
# Load and view first few lines of dataset
df = pd.read_csv('admission_data.csv')
df.head()
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 500 entries, 0 to 499
Data columns (total 4 columns):
student_id    500 non-null int64
gender        500 non-null object
major         500 non-null object
admitted      500 non-null bool
dtypes: bool(1), int64(1), object(2)
memory usage: 12.3+ KB<jupyter_text>Proportion and admission rate for each gender<jupyter_code># get number of males and females in list
gender_count = df['gender'].value_counts()
 
#get total number of values in gender column in variable
gender_total = df['gender'].count()
gender_count
# Proportion of students that are female
print("Proportion of females is {}%".format(100*gender_count[0]/gender_total))
print("Proportion of females is {}%".format(gender_count[0]/gender_total))
# Proportion of students that are male
print("Proportion of males is {}%".format(100*gender_count[1]/gender_total))
print("Proportion of males is {}%".format(gender_count[1]/gender_total))
#To view statistics based on gender, we group by gender variable
# Next we use value_counts() function to see the numbers
df.groupby('gender')['admitted'].value_counts()
# count of females
# https://chrisalbon.com/python/pandas_selecting_rows_on_conditions.html
female_df = df['gender'] == "female"
female_total = sum(female_df)
print("Number of female students are {}".format(female_total))
#count of males
male_df = df['gender'] == "male"
male_total = sum(male_df)
print("Number of male students are {}".format(male_total))
# Admission rate for females
female_admin_rate = 100* 74/female_total
print("Female admission rate is {}%".format(round(female_admin_rate),2))
# Admission rate for males
male_admin_rate = 100* 118/male_total
print("Male admission rate is {}%".format(round(male_admin_rate,2)))
print('Total admission proportions')
print("Female: {}".format(74/257))
print("Male: {}".format(118/243))<jupyter_output>Total admission proportions
Female: 0.28793774319066145
Male: 0.48559670781893005<jupyter_text>Proportion and admission rate for physics majors of each gender<jupyter_code># group by major and gender to get counts of students in each cohort
major_gender_counts = df.groupby(['major', 'gender']).describe()
major_gender_counts.info()
# What proportion of female students are majoring in physics?

# First, get count of females in Physics major 
fem_phy = len(df.loc[(df['gender'] == 'female') & (df['major'] == 'Physics')])
print("Number of females with Physics major is {}".format(fem_phy))

#now evaluate proportion of female candidates in Phsics
print("Proportion of female candidates in Physics is {}%".format(round((fem_phy*100/female_total),2)))
# What proportion of male students are majoring in physics?

# first,get count of males in Physics major 
m_phy = len(df.loc[(df['gender'] == 'male') & (df['major'] == 'Physics')])
print("Number of males with Physics major is {}".format(m_phy))

#now evaluate proportion of male candidates in Physics
print("Proportion of male candidates in Physics is {}%".format(round((m_phy*100/male_total),2)))
# Admission rate for female physics majors
# compute count of females in Physics who got admitted
fem_phy_admit = len(df.loc[(df['gender'] == 'female') 
                           & (df['major'] == 'Physics') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted female candidates in Phsics
print("Proportion of female candidates admitted in Physics is {}%".format(round((fem_phy_admit*100/fem_phy),2)))
# Admission rate for male physics majors
# compute count of females in Physics who got admitted
m_phy_admit = len(df.loc[(df['gender'] == 'male') 
                           & (df['major'] == 'Physics') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted male candidates in Phsics
print("Proportion of male candidates admitted in Physics is {}%".format(round((m_phy_admit*100/m_phy),2)))<jupyter_output>Proportion of male candidates admitted in Physics is 51.56%<jupyter_text>Proportion and admission rate for chemistry majors of each gender<jupyter_code># What proportion of female students are majoring in chemistry?

# First, get count of females in Chemistry major 
fem_chem = len(df.loc[(df['gender'] == 'female') & (df['major'] == 'Chemistry')])
print("Number of females with Chemistry major is {}".format(fem_chem))

#now evaluate proportion of female candidates in Chemistry
print("Proportion of female candidates in Chemistry is {}%".format(round((fem_chem*100/female_total),2)))
# What proportion of male students are majoring in chemistry?
# First, get count of females in Chemistry major 
m_chem = len(df.loc[(df['gender'] == 'male') & (df['major'] == 'Chemistry')])
print("Number of males with Chemistry major is {}".format(m_chem))

#now evaluate proportion of male candidates in Chemistry
print("Proportion of male candidates in Chemistry is {}%".format(round((m_chem*100/male_total),2)))
# Admission rate for female chemistry majors

# compute count of females in chemistry who got admitted
fem_chem_admit = len(df.loc[(df['gender'] == 'female') 
                           & (df['major'] == 'Chemistry') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted female candidates in chemistry
print("Proportion of female candidates admitted in chemistry is {}%".format(round((fem_chem_admit*100/fem_chem),2)))
# Admission rate for male chemistry majors
# compute count of males in chemistry who got admitted
m_chem_admit = len(df.loc[(df['gender'] == 'male') 
                           & (df['major'] == 'Chemistry') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted male candidates in chemistry
print("Proportion of male candidates admitted in chemistry is {}%".format(round((m_chem_admit*100/m_chem),2)))<jupyter_output>Proportion of male candidates admitted in chemistry is 11.11%<jupyter_text>Admission rate for each major<jupyter_code># Admission rate for physics majors
# compute count of students in Physics who got admitted
phy_admit = len(df.loc[(df['major'] == 'Physics') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted candidates in Phsics
print("Proportion of candidates admitted in Physics is {}%".format(round((phy_admit*100/(fem_phy+m_phy)),2)))
# Admission rate for chemistry majors
# compute count of students in Chemistry who got admitted
chem_admit = len(df.loc[(df['major'] == 'Chemistry') 
                           & (df['admitted'] == True)])

#now evaluate proportion of admitted candidates in Chemistry
print("Proportion of candidates admitted in Chemistry is {}%".format(round((chem_admit*100/(fem_chem+m_chem)),2)))<jupyter_output>Proportion of candidates admitted in Chemistry is 21.72%<jupyter_text>Testing: Double check vlaues by computing non-admittance values<jupyter_code># double check values for physics admittance
if (phy_admit ==  (fem_phy_admit+ m_phy_admit)):
    print("Values computed for Physics admitted candidates is Correct")
else: 
    print("Values computed for Physics admitted candidates is Incorrect")


# double check values for Chemistry admittance
if (chem_admit ==  (fem_chem_admit+ m_chem_admit)):
    print("Values computed for Chemistry admitted candidates is Correct")
else: 
    print("Values computed for Chemistry admitted candidates is Incorrect")

# double check not admitted candidates
# candidates not admitted to Physics
not_admit_p = len(df.loc[(df['major'] == 'Physics') 
                           & (df['admitted'] == False)])
# candidates not admitted to Chemistry
not_admit_c = len(df.loc[(df['major'] == 'Chemistry') 
                           & (df['admitted'] == False)])
# compare if admitted and non-admitted candidate counts equal total number of students
if (gender_total ==  (not_admit_p+ not_admit_c+chem_admit+phy_admit)):
    print("Values computed for admitted candidates is Correct")
else: 
    print("Values computed for admitted candidates is Incorrect")<jupyter_output>Values computed for admitted candidates is Correct
{'fraction_non_alphanumeric': 0.11767531219980788, 'fraction_numerical': 0.01825168107588857, 'mean_word_length': 4.915482954545454, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '637308', 'license': 'Apache-2.0', 'n_tokens_mistral': 2591, 'n_tokens_neox': 2381, 'n_words': 806, 'path': 'week3/day5/exercises/2.Simpsons_paradox.ipynb', 'repo_name': 'JavierOlcoz/data_science_nov_2020', 'size': '8328'}