<jupyter_start><jupyter_text>Processing Baby Names Data by YearThis examples uses the baby names by year from the Social Security Administration, available at https://www.ssa.gov/oact/babynames/limits.htmlThe national data file was downloaded and placed in a babynames directory for use.  That directory contains files named yob.txt where  is the year of birth.  Each file is a comma-delimited list with name,sex,occurences.  Order is by occurences, sorted alphabetically on ties.  Outline1. Get list of file names to process2. Read in a file into a dataframe3. Add the year as a column to the dataframe4. accumulate all files, each as a dataframe5. merge the dataframes for opeartion  Get input file listing<jupyter_code>from pathlib import Path
p = Path('./babynames')
raw_list = [pfile for pfile in p.iterdir() if pfile.is_file()]
file_list = [i for i in raw_list if ('yob' in i.name)]
file_list[:5]
### Iterate through files reading them in
import pandas as pd
import numpy as np
df_list = []
header = ['name', 'sex', 'births']
for nfile in file_list:
    year = np.int(nfile.name.strip('.txt').strip('yob'))
    # Note assign here shortcuts a step.
    df_list.append(pd.read_csv(nfile, names=header).assign(yob=year))
df = pd.concat(df_list)
boys = df[df['sex']=='M']
girls = df[df['sex']=='F']
boy_names = boys.groupby('name').sum().sort_values('births', ascending=False)
boy_names = boy_names.reset_index().drop('yob', axis=1)
boy_names.head()
just_aaron = boys[boys['name']=='Aaron']
just_aaron
just_aaron = just_aaron.set_index(just_aaron['yob']).drop(['name', 'sex', 'yob'], axis=1)
just_aaron.plot()
year_totals = boys.groupby('yob').sum()
year_totals
just_aaron['percent'] = just_aaron['births'].values.reshape((139,1))/year_totals.values
just_aaron['percent'].plot()
just_aaron.loc[1979]
just_aaron.loc[1977:1982]<jupyter_output><empty_output><jupyter_text>The Great Hillary Drop<jupyter_code>just_hill = girls[girls['name']=='Hillary']
just_hill = just_hill.set_index(just_hill['yob']).drop(['name', 'sex', 'yob'], axis=1)
just_hill.plot()
just_hill.loc[1980:2000].plot()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13988657844990549, 'fraction_numerical': 0.015595463137996219, 'mean_word_length': 6.176271186440678, 'pattern_counts': {'":': 0, '<': 9, '<?xml version=': 0, '>': 9, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '570012', 'license': 'MIT', 'n_tokens_mistral': 763, 'n_tokens_neox': 728, 'n_words': 200, 'path': 'baby_names.ipynb', 'repo_name': 'sawyerap/PythonDataViz', 'size': '2116'}