<jupyter_start><jupyter_text>Import packages<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
% matplotlib inline<jupyter_output><empty_output><jupyter_text>Import data into dataframe<jupyter_code>data = pd.read_csv('thanksgiving.csv', encoding="Latin-1")
data.head()
data.shape<jupyter_output><empty_output><jupyter_text>Inspect dataframe columns<jupyter_code>data.columns<jupyter_output><empty_output><jupyter_text>How many respondents celebrate thanksgiving<jupyter_code>data['Do you celebrate Thanksgiving?'].value_counts()<jupyter_output><empty_output><jupyter_text>Drop rows of respondents who don't celebrate thanksgiving<jupyter_code>data_yes = data[data['Do you celebrate Thanksgiving?']=='Yes']
data_yes['Do you celebrate Thanksgiving?'].value_counts()<jupyter_output><empty_output><jupyter_text>Inspect distribution of typical dishes<jupyter_code>data_yes['What is typically the main dish at your Thanksgiving dinner?'].value_counts()<jupyter_output><empty_output><jupyter_text>Display rows for families that have tofurkey<jupyter_code>gravy = 'Do you typically have gravy?'
Tof = data_yes['What is typically the main dish at your Thanksgiving dinner?'] == 'Tofurkey'
data_yes[gravy][Tof]<jupyter_output><empty_output><jupyter_text>analyze families that had pies<jupyter_code># simplify labels
Apple_Pie = 'Which type of pie is typically served at your Thanksgiving dinner? Please select all that apply. - Apple'
Pecan_Pie = 'Which type of pie is typically served at your Thanksgiving dinner? Please select all that apply. - Pecan'
Pumpkin_Pie = 'Which type of pie is typically served at your Thanksgiving dinner? Please select all that apply. - Pumpkin'
Pie_labels = [Apple_Pie, Pecan_Pie, Pumpkin_Pie]
# create boolean series for each pie column
Pie_Bools = [pd.isnull(data_yes[pie]) for pie in Pie_labels]
# create counts of different pies consumed
had_pie = sum(Pie_Bools)
# create boolean for having at least one type of pie
ate_pies = Pie_Bools[0] & Pie_Bools[1] & Pie_Bools[2]
data_yes['ate_pies'] = ate_pies
data_yes.ix[:,'ate_pies'].value_counts()<jupyter_output>/Users/javin/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:12: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy<jupyter_text>Create function to convert age from string to integer<jupyter_code>def ageint(age):
    # split age value, grab first age, then replace '+' with blank space
    try:
        age_split = age.split(' ')
        age_no_plus = age_split[0].replace('+','')
        age_int = int(age_no_plus)

        return age_int
    
    # account for empty values
    except AttributeError:
        return<jupyter_output><empty_output><jupyter_text>Create int_age column with ageint function and inspect<jupyter_code>data_yes['int_age'] = data_yes['Age'].apply(ageint)
data_yes['int_age'].describe()<jupyter_output>/Users/javin/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:1: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  if __name__ == '__main__':
/Users/javin/anaconda/lib/python3.5/site-packages/numpy/lib/function_base.py:3834: RuntimeWarning: Invalid value encountered in percentile
  RuntimeWarning)<jupyter_text>ageint findings These aren't the true ages of survey respondents. The values stem from grabbing the first age from each age interval.  Thus, they are ages on the lower end of each respondents age range.   Create function to convert income from string to integer<jupyter_code>def earnings_int(earning):
    # split income value, grab lower income bound, then remove '$' and ',' 
    try:
        earning_lowend = earning.split(' ')[0]
        earning_clean = earning_lowend.replace('$','')
        earning_clean = earning_clean.replace(',','')
        earning_int = int(earning_clean)
        return earning_int
    
    # account for empty values
    except AttributeError:
        return 
    except ValueError:
        return<jupyter_output><empty_output><jupyter_text>Apply function to create int_income column and inspect<jupyter_code>income_column = 'How much total combined money did all members of your HOUSEHOLD earn last year?'
data_yes['int_income'] = data_yes[income_column].apply(earnings_int)
data_yes['int_income'].describe()<jupyter_output>/Users/javin/anaconda/lib/python3.5/site-packages/ipykernel/__main__.py:2: SettingWithCopyWarning: 
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/indexing.html#indexing-view-versus-copy
  from ipykernel import kernelapp as app
/Users/javin/anaconda/lib/python3.5/site-packages/numpy/lib/function_base.py:3834: RuntimeWarning: Invalid value encountered in percentile
  RuntimeWarning)<jupyter_text>int_income findings Similar to ageint, these aren't the true incomes of survey respondents. The values stem from grabbing the first income value from each interval.  Thus, they are incomes on the lower end of each respondents age range. This undervalues the descriptive statistics in the above cell.  Inspect travel distance of income groups<jupyter_code>higher_income = data_yes['int_income'] > 150000
lower_income = data_yes['int_income'] < 150000
data_yes.ix[higher_income,'How far will you travel for Thanksgiving?'].value_counts()
data_yes.ix[higher_income,'How far will you travel for Thanksgiving?'].value_counts(normalize=True)
data_yes.ix[lower_income,'How far will you travel for Thanksgiving?'].value_counts()
data_yes.ix[lower_income,'How far will you travel for Thanksgiving?'].value_counts(normalize=True)<jupyter_output><empty_output><jupyter_text>Travel distance compared to income findings looking at value counts alone, the lower income group takes up a large majority of total respondents. With that in mind, I normalized the counts. When normalized, some similar trends followed for each "How for will you travel..." group. For example, A majority of respondents from both groups had thanksgiving at home, and wouldn't travel at all.  A slight difference emerged, in willingess to travel longer distances. 11.7% of the higher income group were willing to drive several hours, or fly, compared to 7.9% in the lower income group. This distribution shows that 3.8% more likely to travel longer distances for dinner. This however, is a slight difference, and only based on one survey. Thus, it wouldn't be safe to significantly extrapolate this to larger populations. Further statistical analysis would need to be done, to confirm if this difference is event statistically significant.<jupyter_code>data_yes.pivot_table(index="Have you ever tried to meet up with hometown friends on Thanksgiving night?", columns='Have you ever attended a "Friendsgiving?"', values="int_age")
data_yes.pivot_table(index="Have you ever tried to meet up with hometown friends on Thanksgiving night?", columns='Have you ever attended a "Friendsgiving?"', values="int_income")<jupyter_output><empty_output><jupyter_text>Hometown friends and Friendsgiving findings The average age of respondents meeting up with friends and attending thanksgiving were lower. on the opposite end, average age of respondents who refrained from these activities, were higher. It'd seem that younger respondents lean more towards friends for thanksgiving celebrations. added, to that, older respondents may be more focused on their families, which by an older age, should be more established. Average income followed a similar trend, directionally. lower average incomes matched groups that both attended friendsgivings and tried to meet up with friends. Higher average incomes matched groups who did neither.  All together, a trend worth deeper investigation emerges. It would seem that those who celebrate thanksgiving, focus on incorporating freindships at early ages, around times when their income is lower. as they grow older, incomes tend to be higher, and festivities are more focused on family gatherings. Of course, additional samples (survey's conducted) would be needed to confirm these findings with greater confidence.  Investigate deserts consumed<jupyter_code># find index of columns that hold desert values
print(np.where(data_yes.columns.str.contains('desserts')))
dessert_cols = data_yes.columns.str.contains('desserts')
print(dessert_cols)
# find top dessert choices consumed
data_yes.ix[:,dessert_cols].describe().ix['count',:].sort_values(ascending=False)
# display value counts of unique dessert combinations
data_yes.ix[:,dessert_cols].dropna(axis=0,how="all").apply(lambda x: ','.join(x.dropna()), axis=1).value_counts()
data_yes.columns<jupyter_output><empty_output><jupyter_text>Analyze region spread of respondents<jupyter_code># create region counts series
region_counts = data_yes['US Region'].value_counts()
# visualize counts
region_counts.plot(kind='bar')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08353887399463807, 'fraction_numerical': 0.005254691689008043, 'mean_word_length': 6.001501501501502, 'pattern_counts': {'":': 0, '<': 63, '<?xml version=': 0, '>': 63, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '16', 'dir': 'jupyter-structured-clean-dedup', 'id': '280711', 'license': 'MIT', 'n_tokens_mistral': 2810, 'n_tokens_neox': 2665, 'n_words': 921, 'path': 'Analyzing Thanksgiving Dinner/analysis.ipynb', 'repo_name': 'javking07/DataFolio', 'size': '9325'}