<jupyter_start><jupyter_text>Using Boxplots<jupyter_code>import transportation_tutorials as tt
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>Questions 1. Use a boxplot to show the distribution of household income in the Jupiter study area,    by number of automobiles owned.  What is the median income of households who own    exactly two automobiles?  *(Hint: the correct answer is \$96 thousand.)*2. Is the median income higher or lower if we only consider two-car households that   have at least one person over age 65?  Create a set of box plots similar to those   created in question (1), but only for households with at least one person over age 65.  DataTo answer the questions, use the following data files:<jupyter_code>hh = pd.read_csv(tt.data('SERPM8-BASE2015-HOUSEHOLDS'))
hh.head()
person = pd.read_csv(tt.data('SERPM8-BASE2015-PERSONS'))
person.head()<jupyter_output><empty_output><jupyter_text>Solution To answer the first question, we create a boxplot using two columns in `hh` dataframe : `income` and `autos`. But before that, let's filter the records to remove those where incomes are reported as zero or unavailable.<jupyter_code>hh = hh[(hh.income > 0) & (hh.income != 0)]<jupyter_output><empty_output><jupyter_text>The we can draw the boxplot.<jupyter_code>hh_box_1 = list(hh.groupby('autos').income)
plt.boxplot(
    [i[1] for i in hh_box_1], 
    vert = False, 
    labels = [i[0] for i in hh_box_1], 
    notch = True, 
    widths = 0.6,
);<jupyter_output><empty_output><jupyter_text>This is a bit hard to read, as the long tails of very high income households are filling up the figure.We can fix this by clipping the x-axis range, and we can also add some labels to explain whatwe're looking at.<jupyter_code>plt.boxplot(
    [i[1] for i in hh_box_1], 
    vert = False, 
    labels = [i[0] for i in hh_box_1], 
    notch = True, 
    widths = 0.6,
)
plt.xlim(-10000, 200000)
plt.title('Household Income by Number of Autos')
plt.xlabel('Household Income')
plt.ylabel('Autos');<jupyter_output><empty_output><jupyter_text>From this figure, we can more clearly see the median income of households with 2 autos is slightly lower than \$100,000. The actual value of \$96,000 can be found more exactly using the `median` function.<jupyter_code>np.median(hh[hh.autos == 2].income)<jupyter_output><empty_output><jupyter_text>Now, let's move on to the second question. First, we use `person` dataframe to calculate total maximum age for each household using the `.groupby()` method.<jupyter_code>max_age = person.groupby(['hh_id'])['age'].max().rename('max_age')<jupyter_output><empty_output><jupyter_text>Then, we merge the ages with household information to obtain our final dataset and perform some basic filtering to satisfy the conditions.<jupyter_code>hh_2 = pd.merge(hh, max_age, on='hh_id', how='left')
hh_2 = hh_2[(hh_2.max_age > 65) & (hh_2.income > 0)]
hh_2.head()<jupyter_output><empty_output><jupyter_text>We are now ready to prepare the boxplot and find the answer.<jupyter_code>hh_box_2 = list(hh_2.groupby('autos').income)
plt.boxplot(
    [i[1] for i in hh_box_2], 
    vert = False, 
    labels = [str(i[0])+"*" for i in hh_box_2], 
    notch = True, 
    widths = 0.6,
)
plt.xlim(-10000,200000)
plt.title('Household Income by Number of Autos')
plt.xlabel('Household Income')
plt.ylabel('Autos');<jupyter_output><empty_output><jupyter_text>The median income of households who have 2 cars *and* have a person over 65 is less than for all households with 2 cars.The exact value can be confirmed numerically:<jupyter_code>hh_2[hh_2.autos == 2].income.median()<jupyter_output><empty_output><jupyter_text>While more complicated to draw, iterleaving both sets of boxplotsin the same plot makes comparisons between the full population andthe set of households with at least one person over 65 even easier to see:<jupyter_code>plt.boxplot(
    [i[1] for i in hh_box_1], 
    vert = False, 
    labels = [i[0] for i in hh_box_1], 
    notch = True, 
    widths = 0.6,
    positions = range(2, (len(hh_box_2))*2+2, 2),
)

hh_box_2 = list(hh_2.groupby('autos').income)
plt.boxplot(
    [i[1] for i in hh_box_2], 
    vert = False, 
    labels = [str(i[0])+"*" for i in hh_box_2], 
    notch = True, 
    widths = 0.6,
    positions = range(1, (len(hh_box_2))*2+1, 2),
    boxprops=dict(color="red"),
    whiskerprops=dict(color="red"),
    capprops=dict(color="red"),
    flierprops=dict(markeredgecolor="red"),
)
plt.xlim(-10000,200000)
plt.title('Household Income by Number of Autos')
plt.xlabel('Household Income')
plt.ylabel('Autos');<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11370950439819781, 'fraction_numerical': 0.0278910105127655, 'mean_word_length': 4.923761118170267, 'pattern_counts': {'":': 0, '<': 45, '<?xml version=': 0, '>': 48, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '239926', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 1735, 'n_tokens_neox': 1587, 'n_words': 504, 'path': 'course-content/exercises/solution-boxplots.ipynb', 'repo_name': 'chrisc20042001/python-for-transportation-modeling', 'size': '4661'}