<jupyter_start><jupyter_text>This notebook tests whether the Metropolitan Statistical Area (MSA) makes sense. Here, you just plop out which MSA to visualize, say "chicago" or "nyc". It will spit out a figure showing the counties, with FIPS numbers, for that metro region. Looks like I have all the counties defined for the NYC Metro Area. After git commit [fd664d](https://github.com/tanimislam/covid19_stats/commit/fd664daaec025943e4b550c981ae3ea3aace6a28), New York City is a county in the "nyc" metro area with FIPS  "00001".<jupyter_code>fig = pylab.figure()
fig.set_size_inches([18,12])
ax = viz.display_msa( 'nyc', fig, doShow=True )
for patch in ax.patches:
    patch._alpha = 0.2<jupyter_output><empty_output><jupyter_text>Now collect the incident data for the CONUS, and store into the file. Subsequent operations will just *load* this serialized file into this notebook.<jupyter_code>cdat = COVID19Database( )
inc_data_conus = core.get_incident_data( cdat.data_conus( ) )
pickle.dump( inc_data_conus, gzip.open( 'data_conus_LATEST.pkl.gz', 'wb') )<jupyter_output><empty_output><jupyter_text>In subsequent operations, just **load** the `inc_data_conus` from the file.<jupyter_code>inc_data_conus = pickle.load( gzip.open( 'data_conus_LATEST.pkl.gz', 'rb'))<jupyter_output><empty_output><jupyter_text>This is the number of cases and deaths, *cumulatively*, in the [CONUS](https://en.wikipedia.org/wiki/Contiguous_United_States)<jupyter_code>cumul_cases = numpy.array( inc_data_conus['df'].cases, dtype=int )
cumul_deaths = numpy.array( inc_data_conus['df'].death, dtype=int )<jupyter_output><empty_output><jupyter_text>These are the days-in-order from the first incident day. For the 7-day average, start on *seventh* day after first incident day. For the 1-day average, start on the *first* day after first incident day.<jupyter_code>num_days  = len( numpy.array(inc_data_conus['df'].cases ) )
days_7day = numpy.array( list(range( 7, num_days ) ), dtype=int )
days_1day = numpy.array( list(range( 1, num_days ) ), dtype=int )<jupyter_output><empty_output><jupyter_text>Here are the 7 day and 1 day COVID-19 case and death rates for the `CONUS`<jupyter_code>cases_rate_7day = ( cumul_cases[7:] - cumul_cases[:-7] ) / 7.0
cases_rate_1day = ( cumul_cases[1:] - cumul_cases[:-1] ) / 1.0
death_rate_7day = ( cumul_deaths[7:] - cumul_deaths[:-7] ) / 7.0
death_rate_1day = ( cumul_deaths[1:] - cumul_deaths[:-1] ) / 1.0<jupyter_output><empty_output><jupyter_text>Plot of the 7 day and 1 day averaged new COVID-19 case rate for the `CONUS`.<jupyter_code>fig, ax = pylab.subplots( )
fig.set_size_inches([18,12])
#
ax.plot( days_7day, cases_rate_7day, linewidth = 4.5, label = '7 day average' )
ax.plot( days_1day, cases_rate_1day, alpha = 0.2, linewidth = 4.5, label = '1 day average' )
#
ax.set_xlim(7, len(numpy.array(inc_data_conus['df'].cases, dtype=float ) ) - 1 )
ax.set_ylim(0, 3e5)
ax.set_xlabel( 'days after the first incident', fontsize = 18, fontweight = 'bold' )
ax.set_ylabel( 'averaged new COVID-19 cases/day', fontsize = 18, fontweight = 'bold')
for tick in ax.xaxis.get_ticklabels() + ax.yaxis.get_ticklabels( ):
    tick.set_fontsize( 16 )
    tick.set_fontweight( 'bold')
ax.ticklabel_format(axis='y', scilimits=(0,0), useMathText = True)
leg = ax.legend( )
for txt in leg.texts:
    txt.set_fontsize( 20 )
    txt.set_fontweight( 'bold' )
ax.set_title( 'AVERAGED COVID-19 NEW CASES RATES FOR CONUS', fontsize = 24, fontweight = 'bold')
fig, ax = pylab.subplots( )
fig.set_size_inches([18,12])
ax.plot( days_7day, death_rate_7day, linewidth = 4.5, label = '7 day average' )
ax.plot( days_1day, death_rate_1day, alpha = 0.2, linewidth = 4.5, label = '1 day average' )
ax.set_xlim(7, len(numpy.array(inc_data_conus['df'].cases, dtype=float ) ) - 1 )
ax.set_ylim(0, 5000)
ax.set_xlabel( 'days after the first incident', fontsize = 18, fontweight = 'bold' )
ax.set_ylabel( 'averaged new COVID-19 deaths/day', fontsize = 18, fontweight = 'bold')
for tick in ax.xaxis.get_ticklabels() + ax.yaxis.get_ticklabels( ):
    tick.set_fontsize( 16 )
    tick.set_fontweight( 'bold')
#ax.ticklabel_format(axis='y', scilimits=(0,0), useMathText = True)
leg = ax.legend( )
for txt in leg.texts:
    txt.set_fontsize( 20 )
    txt.set_fontweight( 'bold' )
ax.set_title( 'AVERAGED COVID-19 NEW DEATHS RATES FOR CONUS', fontsize = 24, fontweight = 'bold')
inc_data_bayarea = core.get_incident_data( cdat.data_msas_2019( )[ 'bayarea' ] )
cumul_cases_bayarea = numpy.array( inc_data_bayarea['df'].cases, dtype=int )
cumul_deaths_bayarea = numpy.array( inc_data_bayarea['df'].death, dtype=int )
num_days_bayarea  = len( numpy.array(inc_data_bayarea['df'].cases ) )
days_7day_bayarea = numpy.array( list(range( 7, num_days_bayarea ) ), dtype=int )
days_1day_bayarea = numpy.array( list(range( 1, num_days_bayarea ) ), dtype=int )
cases_rate_7day_bayarea = ( cumul_cases_bayarea[7:] - cumul_cases_bayarea[:-7] ) / 7.0
cases_rate_1day_bayarea = ( cumul_cases_bayarea[1:] - cumul_cases_bayarea[:-1] ) / 1.0
death_rate_7day_bayarea = ( cumul_deaths_bayarea[7:] - cumul_deaths_bayarea[:-7] ) / 7.0
death_rate_1day_bayarea = ( cumul_deaths_bayarea[1:] - cumul_deaths_bayarea[:-1] ) / 1.0
fig, ax = pylab.subplots( )
fig.set_size_inches([18,12])
#
ax.plot( days_7day_bayarea, cases_rate_7day_bayarea, linewidth = 4.5, label = '7 day average' )
ax.plot( days_1day_bayarea, cases_rate_1day_bayarea, alpha = 0.2, linewidth = 4.5, label = '1 day average' )
#
ax.set_xlim(7, len(numpy.array(inc_data_bayarea['df'].cases, dtype=float ) ) - 1 )
ax.set_ylim(0, 5000)
ax.set_xlabel( 'days after the first incident', fontsize = 18, fontweight = 'bold' )
ax.set_ylabel( 'averaged new COVID-19 cases/day', fontsize = 18, fontweight = 'bold')
for tick in ax.xaxis.get_ticklabels() + ax.yaxis.get_ticklabels( ):
    tick.set_fontsize( 16 )
    tick.set_fontweight( 'bold')
#ax.ticklabel_format(axis='y', scilimits=(0,0), useMathText = True)
leg = ax.legend( )
for txt in leg.texts:
    txt.set_fontsize( 20 )
    txt.set_fontweight( 'bold' )
ax.set_title( 'AVERAGED COVID-19 NEW CASE RATES FOR BAY AREA', fontsize = 24, fontweight = 'bold')
inc_data_bayarea['df'].date.max( ), inc_data_bayarea['df'].date.min( ), int( cases_rate_7day_bayarea[-1] )
# ax.text( 25.0, 4000, 'FIRST', fontdict = { 'fontsize' : 20, 'fontweight' : 'bold' } )
fig, ax = pylab.subplots( )
fig.set_size_inches([18,12])
ax.plot( days_7day_bayarea, death_rate_7day_bayarea, linewidth = 4.5, label = '7 day average' )
ax.plot( days_1day_bayarea, death_rate_1day_bayarea, alpha = 0.2, linewidth = 4.5, label = '1 day average' )
ax.set_xlim(7, len(numpy.array(inc_data_bayarea['df'].cases, dtype=float ) ) - 1 )
ax.set_ylim(0, 120 )
ax.set_xlabel( 'days after the first incident', fontsize = 18, fontweight = 'bold' )
ax.set_ylabel( 'averaged new COVID-19 deaths/day', fontsize = 18, fontweight = 'bold')
for tick in ax.xaxis.get_ticklabels() + ax.yaxis.get_ticklabels( ):
    tick.set_fontsize( 16 )
    tick.set_fontweight( 'bold')
#ax.ticklabel_format(axis='y', scilimits=(0,0), useMathText = True)
leg = ax.legend( )
for txt in leg.texts:
    txt.set_fontsize( 20 )
    txt.set_fontweight( 'bold' )
ax.set_title( 'AVERAGED COVID-19 NEW DEATH RATES FOR BAY AREA', fontsize = 24, fontweight = 'bold')
from dateutil.relativedelta import relativedelta
relativedelta(days=340) + inc_data_bayarea['df'].date.min( )<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14109681787406905, 'fraction_numerical': 0.03628977657413676, 'mean_word_length': 5.445026178010472, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 29, 'https://': 2, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '459982', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 3091, 'n_tokens_neox': 2846, 'n_words': 642, 'path': 'testing/visualize MSA.ipynb', 'repo_name': 'tanimislam/covid19_stats', 'size': '7385'}