<jupyter_start><jupyter_text>Part 1 -- Intro Load the tips dataset from seaborn<jupyter_code>import seaborn as sns

tips = sns.load_dataset('tips')
tips.head()<jupyter_output><empty_output><jupyter_text>Filter rows by `smoker == 'No'` and `total_bill >= 10`<jupyter_code>tips.loc[(tips["smoker"] == "No") & (tips.total_bill >= 10)]<jupyter_output><empty_output><jupyter_text>What is the average `total_bill` for each value of `smoker`, `day`, and `time`<jupyter_code>tips.groupby(['smoker', 'day', 'time'])['total_bill'].mean().reset_index()<jupyter_output><empty_output><jupyter_text>Part 2 -- Tidy Taken from the r4ds "Tidy Data" Chapter: https://r4ds.had.co.nz/exploratory-data-analysis.html<jupyter_code>import pandas as pd
tbl1 = pd.read_csv('../data/table1.csv')
tbl2 = pd.read_csv('../data/table2.csv')
tbl3 = pd.read_csv('../data/table3.csv')
tbl1<jupyter_output><empty_output><jupyter_text>Tidy the `tbl2` dataset<jupyter_code>tbl2
tbl2.pivot_table(index=['country', 'year'],
                columns='type',
                values='count').reset_index()<jupyter_output><empty_output><jupyter_text>Tidy the `tbl3` dataset<jupyter_code># just give me the population
tbl3
tbl3['population'] = tbl3['rate'].str.split('/').str.get(1)
tbl3<jupyter_output><empty_output><jupyter_text>Part 3 -- Apply functions Look at the `table3` dataset<jupyter_code>tbl3 = pd.read_csv('../data/table3.csv')
tbl3<jupyter_output><empty_output><jupyter_text>Write a function that takes a value of `rate` and parses out the total population.<jupyter_code>tbl3.dtypes
def extract_population(rate, delim='/', position=1):
    pop = rate.split(delim)[position]
    return int(pop)
assert extract_population('123/456') == 456
pops = tbl3['rate'].apply(extract_population)<jupyter_output><empty_output><jupyter_text>Set the population to a new column<jupyter_code>tbl3['pop'] = pops
tbl3
tbl3['pop2'] = tbl3['rate'].apply(extract_population)
tbl3<jupyter_output><empty_output><jupyter_text>Part 4 -- Plots<jupyter_code>titanic = sns.load_dataset('titanic')
titanic.head()<jupyter_output><empty_output><jupyter_text>Create a figure with 2 axes distplot of `fare` in one axes boxplot of `class` and `fare` on the other axes<jupyter_code>import matplotlib.pyplot as plt
import seaborn as sns
fig, (ax1, ax2) = plt.subplots(1, 2)
sns.distplot(titanic.fare, ax=ax1)
sns.boxplot(x='class', y='fare', data=titanic, ax=ax2)<jupyter_output><empty_output><jupyter_text>Part 5 -- Models<jupyter_code>titanic = sns.load_dataset('titanic')
titanic.head()
titanic.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 891 entries, 0 to 890
Data columns (total 15 columns):
survived       891 non-null int64
pclass         891 non-null int64
sex            891 non-null object
age            714 non-null float64
sibsp          891 non-null int64
parch          891 non-null int64
fare           891 non-null float64
embarked       889 non-null object
class          891 non-null category
who            891 non-null object
adult_male     891 non-null bool
deck           203 non-null category
embark_town    889 non-null object
alive          891 non-null object
alone          891 non-null bool
dtypes: bool(2), category(2), float64(2), int64(4), object(5)
memory usage: 80.6+ KB<jupyter_text>Subset `survived`, `class`, `who`<jupyter_code>titanic_subset = titanic[["survived", "class", "who"]]<jupyter_output><empty_output><jupyter_text>Create dummy encoded dataset<jupyter_code>titanic_dummy = pd.get_dummies(titanic_subset, drop_first=True)
titanic_dummy.head()<jupyter_output><empty_output><jupyter_text>Fit a logistic regression on `survived`<jupyter_code>from sklearn.linear_model import LogisticRegression
X = titanic_dummy.iloc[:, 1:]
y = titanic_dummy.iloc[:, 0]
clf = LogisticRegression(random_state=0, solver='lbfgs', multi_class='multinomial').fit(X, y)
clf.coef_<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13525327847775778, 'fraction_numerical': 0.03522756492671638, 'mean_word_length': 5.683848797250859, 'pattern_counts': {'":': 0, '<': 61, '<?xml version=': 0, '>': 63, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '184662', 'license': 'MIT', 'n_tokens_mistral': 1534, 'n_tokens_neox': 1432, 'n_words': 287, 'path': 'exercises/exercises.ipynb', 'repo_name': 'seanreed1111/scipy-2019-pandas', 'size': '3889'}