<jupyter_start><jupyter_text>Week 3-2 - Logistic Regression - homeworkIn this assignment, you'll do a logistic regression to look at the effect of sex and class on survival on the titanic, by computing odds ratios.Some references:- [What are odds vs. probability?](https://towcenter.gitbooks.io/curious-journalist-s-guide-to-data/content/analysis/counting_possible_worlds.html)- An example of a story based on an odds ratio: [Deadly force in black and white](https://www.propublica.org/article/deadly-force-in-black-and-white)- [Building A Logistic Regression in Python, Step by Step](https://towardsdatascience.com/building-a-logistic-regression-in-python-step-by-step-becd4d56c9c8) Here it's being used for prediction by looking at the output, whereas we are using it for explanation by looking at the coefficient, but setting it up works the same.- [How do I interpret odds ratios in logistic regression?](https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-how-do-i-interpret-odds-ratios-in-logistic-regression/) This one's a little more technical, but has good examples.<jupyter_code>import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Load the dataRead in the `titanic.csv` data set again.<jupyter_code># Load titanic.csv
df = pd.read_csv('week-3/titanic.csv')
df.tail()<jupyter_output><empty_output><jupyter_text>The first thing we need to do is code the pclass and gender variables numerically. Let's use the following scheme:- pclass: 1,2,3- gender: 0=male, 1=female, and let's call the column called "female" to remind us which is which<jupyter_code># recode the pclass and gender variables so they are numeric
df.replace({
  '1st': 1,
  '2nd': 2,
  '3rd': 3
}, inplace=True)

df.replace({
  'male': 0,
  'female': 1
}, inplace=True)

df['female'] = df['gender']
df.head()<jupyter_output><empty_output><jupyter_text>2. Looking at one variable at a timeFirst, do a logistic regression of the `female` variable alone. Below is some code that will help you plot the result.<jupyter_code># function to plot single variable logistic regression results
def plot_logistic_regression(x, y, lm):

    # plot original data
    plt.scatter(x=x,y=y)

    # Now plot the prediction of the fitted model
    predict_x = np.linspace(x.min(), x.max(), 10).reshape(-1, 1) # some evenly spaced points to predict
    predict_y = lm.predict_proba(predict_x)[:,1] #  probability of result = true
    plt.plot(predict_x, predict_y, '-', color='m')

# Do a logistic regression of survived against pclass
# Are we doing gender or class now? I'm confused.
x = df[['female']].values
y = df[['survived']].values
lm = LogisticRegression()
lm.fit(x, y)

plot_logistic_regression(x, y, lm)<jupyter_output>/Users/km/.pyenv/versions/3.6.5/lib/python3.6/site-packages/sklearn/utils/validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>The plot may not look like much, because there are only two values of `female` so most of the dots are on top of eachother. But the regression should capture the fact that more women survived than men, by sloping upward. Most of the information will be in the resulting odds ratio. This will tell us how much the odds of surivial differ between men and women. Compute it now.<jupyter_code># What is the odds ratio of on the class variable?
# Again, assuming that this meant to say "female"
# But then again, I don't assume that anyone will ever read this comments, 
# because "grading" only consists of counting how many error messages will 
# pop up in the cells?
np.exp(lm.coef_)<jupyter_output><empty_output><jupyter_text>3. Looking at two variables at a timeWe know from the earlier assignment that class also affects survival, so let's add that to our model. Compute a logistic regression on the variables `pclass` and `female`.<jupyter_code># Logistic regression on two variables
fields = ['female', 'pclass']
x = df[fields].values
y = df[['survived']].values
lm = LogisticRegression()
lm.fit(x,y)<jupyter_output>/Users/km/.pyenv/versions/3.6.5/lib/python3.6/site-packages/sklearn/utils/validation.py:578: DataConversionWarning: A column-vector y was passed when a 1d array was expected. Please change the shape of y to (n_samples, ), for example using ravel().
  y = column_or_1d(y, warn=True)<jupyter_text>Now compute the odds ratio on each variable. You can print it out slightly prettier by using the trick in the class where we changed it into a dataframe.<jupyter_code># print out the odds ratios for each variable
results = pd.DataFrame(np.exp(lm.coef_), columns=fields)
results<jupyter_output><empty_output><jupyter_text>4. What does this meanWhat is the odds ratio on the `pclass` variable? What happens to the odds of survival when we move from 1st to 2nd or from 2nd to 3rd class? The odds ratio on pclass is 0.36, meaning that the odds of survival decrease by about three times when we move from 1st to 2nd class, and again from 2nd to 3rd class. The odds ratio on the `female` variable changed when we accounted for passenger class. Which direction, and what does this mean? The odds ratio on `female` changed from 9.3 to 10.7. This means that when we are careful to compare the surivial rates of the sexes between passengers of the same class, we can see that women have even better odds of survival than it first appears. The number changes because the odds of surviving are lower in lower passenger classes, and those classes have more people in them. So when we only compare passengers within the same class, being female has an even greater advantage.  Bonus: How much effect did opinion of Obamacare have in the 2016 election?Let's look at votes for Trump in the ANES data again. Try it on the following columns:- Who did you vote for? This will be our dependent variable- Do you approve or disapprove of Obamacare?- What political party are you registered for?Start by doing a one variable logistic regression on vote vs. health care. Then add in political party, so that we are comparing the effect of opinion on health care only within voters who are registered to the same party. What happens to the odds ratio on the health care variable when we take into account which party they are registered to? Why?<jupyter_code># The columns we'll use
voted_col = 'V162034a' # 1=Clinton, 2=Trump, 3=Johnson, 4=Stein, negative numbers = didn't vote or won't say
health_care_col = 'V161085' # pre-election survey. 1=approve, 2=dissaprove,  negative numbers = didn't vote or won't say
registered_col = 'V161019' # 1=democratic, 2=republican, 3=none, 4=other, negative = not applicable or refused
# load the data and keep only the columns we need
anes = pd.read_csv('week-3/anes_timeseries_2016_rawdata.csv')
anes.head()
# throw out all rows where:
# - didn't vote Dem or GOP
anes = anes[(anes[voted_col] == 1) | (anes[voted_col] == 2)]

# - wasn't registered demo or GOP
anes = anes[(anes[registered_col] == 1) | (anes[registered_col] == 2)]

# - answer on health care question is not approve or disaprove
anes = anes[(anes[health_care_col] == 1) | (anes[health_care_col] == 2)]

anes.shape

# Recode each variable so it's 0-1 instead of 1-2
anes.replace({
  1: 0,
  2: 1
}, inplace=True)
# Do a single variable logistic regression of vote on health care, and plot
x = anes[[health_care_col]].values
y = anes[[voted_col]].values
lim = LogisticRegression()
lim.fit(x,y)

plot_logistic_regression(x,y,lim)
# What is the odds ratio of voting for Trump?
# That is, starting with the odds of voting for Trump if you approve of Obamacare,  
# how do the odds change if you dissaprove of Obamacare?
np.exp(lim.coef_)
# Do a multiple variable logistic regression of vote on health care and party registration
fields = [health_care_col, registered_col]
x = anes[fields].values
y = anes[[voted_col]].values
lim = LogisticRegression()
lim.fit(x, y)

# What is the odds ratio of voting for Trump if you dissaprove of Obamacare now?
# What is the odds ratio of voting for Trump if you are a registered Republican?
results = pd.DataFrame(np.exp(lm.coef_), columns=fields)
results<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08737397983677389, 'fraction_numerical': 0.013922227556409025, 'mean_word_length': 4.8765867418899855, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 31, 'https://': 4, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '317143', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 2648, 'n_tokens_neox': 2474, 'n_words': 1075, 'path': 'week-3/week-3-2-homework.ipynb', 'repo_name': 'xeophin/lede-algorithms', 'size': '8332'}