<jupyter_start><jupyter_text><NAME>  Research question/interestsIs there a correlation between age and death to covid? Is there a correlation between gender and death to covid?Who is the most suseptible population?  Milestone 3 Task 3: Method Chaining and writing python<jupyter_code>#Import Libraries
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pylab as plt
#Removing limit on how many columns I can see
pd.set_option("display.max_columns", None)
pd.set_option("display.max_rows", None)
#Importing Functions
from scripts.project_functions1 import *
#Settting up data visualization things
%matplotlib inline
%reload_ext autoreload
%autoreload 2
df1 = (
    pd.read_csv("../data/raw/raw_data.csv")
    .iloc[:, np.r_[1,2,46:58,93:105]]
    .iloc[36:,]
    .drop(columns="caseAgeUnknown")
    .drop(columns="deathAgeUnknown")
)
try:
    df2=df_percentage(df1,'case0to9','death0to9','perc0to9')
    df2=df_percentage(df1,'case10to19','death10to19','perc10to19')
    df2=df_percentage(df1,'case20to29','death20to29','perc20to29')
    df2=df_percentage(df1,'case30to39','death30to39','perc30to39')
    df2=df_percentage(df1,'case40to49','death40to49','perc40to49')
    df2=df_percentage(df1,'case50to59','death50to59','perc50to59')
    df2=df_percentage(df1,'case60to69','death60to69','perc60to69')
    df2=df_percentage(df1,'case70to79','death70to79','perc70to79')
    df2=df_percentage(df1,'case80plus','death80plus','perc80plus')
except:
    print("Something wrong in the ages")
try:
    df2=df_percentage(df1,'genMale','deathGenMale','percMale')
    df2=df_percentage(df1,'genFemale','deathGenFemale','percFemale')
except:
    print("Something wring in the genders")

df2.head()
df3=df2.iloc[:, np.r_[0,1,24:35]] #Making a third dataframe for easier visualization
df3.head()<jupyter_output><empty_output><jupyter_text>Task 4: Visualization<jupyter_code>plt.figure(figsize=(8,6))
sns.set_context('paper',font_scale=1.4)

df3_mx = df3.corr()
sns.heatmap(df3_mx, annot=True, cmap ='Blues')<jupyter_output><empty_output><jupyter_text>Second attempt at visualizaiton I wanted to map the change in the percent deaths overtime.<jupyter_code>sns.scatterplot(data=df3, x= 'TotalCases', y='percFemale')
sns.scatterplot(data=df3, x= 'TotalCases', y='percMale')<jupyter_output><empty_output><jupyter_text>This doesn't really didn't hit what I wanted it to so I decided to switch to density plots instead<jupyter_code>fig, ax = plt.subplots()
sns.kdeplot(df3['percFemale'], ax=ax, label='Female')
sns.kdeplot(df3['percMale'], ax=ax,label='Male')
plt.legend()<jupyter_output><empty_output><jupyter_text>What this does is plots the density of percentage so it shows that men were more often at a higher percentage of death meaning that they were more likely to die from the virus.  I wen't through a couple different versions of this for the age groups as putting them all into one looked bad and was difficult to read as seen below:<jupyter_code>fig, ax = plt.subplots()
sns.kdeplot(df3['perc0to9'], ax=ax, label='0 to 9')
sns.kdeplot(df3['perc10to19'], ax=ax,label='10 to 19')
sns.kdeplot(df3['perc20to29'], ax=ax, label='20 to 29')
sns.kdeplot(df3['perc30to39'], ax=ax,label='30 to 39')
sns.kdeplot(df3['perc40to49'], ax=ax, label='40 to 49')
sns.kdeplot(df3['perc50to59'], ax=ax,label='50 to 59')
sns.kdeplot(df3['perc60to69'], ax=ax, label='60 to 69')
sns.kdeplot(df3['perc70to79'], ax=ax,label='70 to 79')
sns.kdeplot(df3['perc80plus'], ax=ax, label='80 +')
plt.legend()<jupyter_output><empty_output><jupyter_text>This is obviously unreasonable so I decided to split them up into subgroups<jupyter_code>fig, ax = plt.subplots()
sns.kdeplot(df3['perc0to9'], ax=ax, label='0 to 9')
sns.kdeplot(df3['perc10to19'], ax=ax,label='10 to 19')
plt.legend()
fig, ax = plt.subplots()
sns.kdeplot(df3['perc20to29'], ax=ax, label='20 to 29')
sns.kdeplot(df3['perc30to39'], ax=ax,label='30 to 39')
sns.kdeplot(df3['perc40to49'], ax=ax, label='40 to 49')
plt.legend()
fig, ax = plt.subplots()
sns.kdeplot(df3['perc50to59'], ax=ax,label='50 to 59')
sns.kdeplot(df3['perc60to69'], ax=ax, label='60 to 69')
plt.legend()
fig, ax = plt.subplots()
sns.kdeplot(df3['perc70to79'], ax=ax,label='70 to 79')
sns.kdeplot(df3['perc80plus'], ax=ax, label='80 +')
plt.legend()<jupyter_output><empty_output><jupyter_text>Basically what all of these suggest is that the older you are, the more likely you are to get covid as the older you get, the larger the percentage is for people who died from the virus. We also determined that men were more likely to die from covid than women but that could be a coincidence more than a causation and requires further study<jupyter_code>#Moving Processed data into a csv
df3.to_csv('processed.csv')
#Manually moved it into the data/processed folder<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.140719185200582, 'fraction_numerical': 0.06339638328829765, 'mean_word_length': 7.21160409556314, 'pattern_counts': {'":': 0, '<': 30, '<?xml version=': 0, '>': 30, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '639610', 'license': 'MIT', 'n_tokens_mistral': 1999, 'n_tokens_neox': 1800, 'n_words': 428, 'path': 'notebooks/analysis1.ipynb', 'repo_name': 'data301-2021-winter1/solo-BilodeauS', 'size': '4820'}