<jupyter_start><jupyter_text>dataquest.io - Python Programming Guided project - explore US BirthsThis notepad analyses the birth rates from the US between 1993 and 2014 using the CDC and SSA data sets that are available [here](https://github.com/fivethirtyeight/data/tree/master/births).The following functions are defined:* `read_csv(file_name)`* `calc_counts(data, column)`* `calc_max_dict_value(dictionary)`* `calc_min_dict_value(dictionary)`* `calc_counts_with_condition(data,condition_col, condition_val, group_by_col,value_col=4)`* `calc_diff_counts(data)`* `calc_diff_counts_with_condition(data,condition_col, condition_val, group_by_col,value_col=4)`* `create_date_key(data)`* `create_dictionary_with_date_key(data)`* `combine_dictionaries(dict_1, dict_2, col_to_merge)`<jupyter_code>def read_csv(file_name):
    file = open(file_name).read()
    data = file.split("\n")
    end_index = len(data)
    string_list = data[1:end_index]
    
    final_list = []
    
    for row in string_list:
        int_fields = []
        string_fields = row.split(",")
        
        for field in string_fields:
            field_value = int(field)
            int_fields.append(field_value)
            
        final_list.append(int_fields)
    
    return final_list  
cdc_list = read_csv("US_births_1994-2003_CDC_NCHS.csv")
cdc_list[0:10]
def calc_counts(data, column):
    final_list = {}
    
    for data_row in data:
        column_value = data_row[column]
        births = data_row[4]
        
        if column_value in final_list:
            final_list[column_value] = final_list[column_value] + births
        else:
            final_list[column_value] = births
        
    return final_list
cdc_year_births = calc_counts(cdc_list, 0)
cdc_month_births = calc_counts(cdc_list, 1)
cdc_dom_births = calc_counts(cdc_list, 2)
cdc_dow_births = calc_counts(cdc_list, 3)
def calc_max_dict_value(dictionary):
      return max(dictionary.values())
    
def calc_min_dict_value(dictionary):
      return min(dictionary.values())<jupyter_output><empty_output><jupyter_text>calc_counts_with_conditionReturns a dictionary from a list of lists that contains the summed values of elements in rows that meet a specified condition.* `condition_col` - the column to check the value of as part of the condition* `condition_val` - the value the condition column value needs to equal to be included in the returned dictionary* `group_by_col` - the column to use as the 'key' for each group of summed values (i.e. 0 = year, 1 = month)* `value_col` - the column containing the value to be used when summing the rows that meet the condition (default = births [4])<jupyter_code>def calc_counts_with_condition(data,condition_col, condition_val, group_by_col,value_col=4):
    grouped_list = {}
    
    for row in data:
        key = row[group_by_col]
        value = row[value_col]
        condition = row[condition_col]
        
        if condition == condition_val:
            if key in grouped_list:
                grouped_list[key] = grouped_list[key] + value
            else:
                grouped_list[key] = value
            
    return grouped_list<jupyter_output><empty_output><jupyter_text>calc_diff_countsCalculates the difference between values within a dictionary from one element to the next. Elements are ordered by the dictionary key<jupyter_code>def calc_diff_counts(data):
    diff_list = {}
    
    sorted_list = sorted(data)
    prev_value = 0
    
    for row in sorted_list:
        diff_list[row] = data[row] - prev_value
        prev_value = data[row]
        
    return diff_list<jupyter_output><empty_output><jupyter_text>Merge two datasets with same structureThe following cells merge to data sets (SSA (2000 to 2014) and CDC (1994 - 2003)) into  a single list.Where the two data sets overlap on a date, the mean of the values within both data sets is uses in place of the original values (decimal places removed by converting float value to int)Each data set is converted to a dictionary with the key being composed of the unique date for each data item, and the value being the data item, alowing matching rows to be found using a straightforward check that a key in one list exists in the other, i.e.[1994, 1, 7, 5, 11251] becomes { **'1994.1.7'**: [1994, 1, 7, 5, 11251]}<jupyter_code>ssa_list = read_csv("US_births_2000-2014_SSA.csv")
def create_date_key(data):
    return str(data[0]) + '.' + str(data[1]) + '.' + str(data[2])

def create_dictionary_with_date_key(data):
    dictionary = {}
    for row in data:
        key = create_date_key(row)
        dictionary[key] = row
    return dictionary

date_keyed_ssa_dictionary = create_dictionary_with_date_key(ssa_list)
date_keyed_cdc_dictionary = create_dictionary_with_date_key(cdc_list)

def merge_dictionaries(dict_1, dict_2, col_to_merge=4):
       
    merged_dict = dict_1
    
    for row in dict_2:
        if row in dict_1:
            value_1 = dict_1[row][col_to_merge]
            value_2 = dict_2[row][col_to_merge]
            mean = int((value_1 + value_2) / 2)
            
            merged_dict[row][col_to_merge] = mean
        else:
            merged_dict[row] = dict_2[row]
           
    return merged_dict

merged_dictionaries = merge_dictionaries(date_keyed_cdc_dictionary,date_keyed_ssa_dictionary)
merged_list = [ v for v in merged_dictionaries.values()]

merged_list[0:10]<jupyter_output><empty_output><jupyter_text>calc_diff_counts_with_conditionCombines the retrieval of values that meet a given condition using the `calc_counts_with_condition` function, with the `calc_diff_counts` function<jupyter_code>def calc_diff_counts_with_condition(data,condition_col, condition_val, group_by_col,value_col=4):
    grouped_conditional_list = calc_counts_with_condition(data,condition_col,condition_val,group_by_col,value_col)
    diff_list = calc_diff_counts(grouped_conditional_list)
    
    return diff_list<jupyter_output><empty_output><jupyter_text>Extracts the difference in all births on Saturdays (specified via `condition_column` & `condition_value`) from one year (specified by `group_by_column`) to the next<jupyter_code>condition_column = 3 # day of week
condition_value = 6 # saturday
group_by_column = 0 # year

diff_list = calc_diff_counts_with_condition(merged_list,condition_column,condition_value,group_by_column)
diff_list<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08130591743839272, 'fraction_numerical': 0.017579657824517345, 'mean_word_length': 4.138709677419355, 'pattern_counts': {'":': 0, '<': 25, '<?xml version=': 0, '>': 25, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '447397', 'license': 'MIT', 'n_tokens_mistral': 2152, 'n_tokens_neox': 2035, 'n_words': 519, 'path': 'Python/Basics/Basics.ipynb', 'repo_name': 'haydnchapman/dataquest.io', 'size': '6371'}