<jupyter_start><jupyter_text>Covid-19 examplesThe data is provided by NYTimes which can be found here: https://github.com/nytimes/covid-19-data.  The data file used here is state-based US data.<jupyter_code>from chart_ipynb.chart_framework import ChartSuperClass
from chart_ipynb import utils, line, time_series, bar, scatter, bubble
import numpy as np
import pandas as pd
from jp_doodle import dual_canvas
from IPython.display import display
import ipywidgets as widgets
url = 'https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-states.csv'
data = pd.read_csv(url)
data.head()<jupyter_output><empty_output><jupyter_text>First, we define a function called get_state_data to extract information based on states.<jupyter_code>def get_state_data(data, state, start=None, end=None):
    """
    data: pd.DataFrame
    state: a str of state or a list of states
    """
    if start is None:
        start = data.iloc[0].date
    if end is None:
        end = data.iloc[-1].date
    data = data[(data.date >= start) & (data.date <= end)]
    states = data.reset_index().groupby('state')
    if isinstance(state, str):
        state = [state]
    state_data = dict()
    for s in state:
        idx = states.groups[s]
        state_data[s] = data.iloc[idx].reset_index(drop=True)
    return state_data
state_name = ['California', 'Texas', 'Florida', 'New York', 'Pennsylvania', 'Illinois', 'Ohio', 'Georgia', 'North Carolina', 'Michigan', 'New Jersey', 'Virginia', 'Washington', 'Arizona', 'Massachusetts', 'Tennessee', 'Indiana', 'Missouri', 'Maryland', 'Wisconsin', 'Colorado', 'Minnesota', 'South Carolina', 'Alabama', 'Louisiana', 'Kentucky', 'Oregon', 'Oklahoma', 'Connecticut', 'Utah', 'Iowa', 'Nevada', 'Arkansas', 'Puerto Rico', 'Mississippi', 'Kansas', 'New Mexico', 'Nebraska', 'Idaho', 'West Virginia', 'Hawaii', 'New Hampshire', 'Maine', 'Montana', 'Rhode Island', 'Delaware', 'South Dakota', 'North Dakota', 'Alaska', 'District of Columbia', 'Vermont', 'Wyoming']<jupyter_output><empty_output><jupyter_text>Covid-19 Cases by StatesWe can compare the number of Covid-19 cases of New York and New Jersey starting from March 1st.<jupyter_code>state = ['New York', 'New Jersey']
start = '2020-03-01'
state_data = get_state_data(data, state, start = start)
input_dataset = [state_data[s] for s in state]<jupyter_output><empty_output><jupyter_text>`time_series_Chart` support two types of charts: line and bar. More details about time series function can be found [here](https://github.com/AaronWatters/Chart_ipynb/blob/master/notebooks/time%20series%20example.ipynb). Line Chart<jupyter_code>time_series.time_series_Chart('line', state, 'cases', date_col = 'date', colors = ['red', 'blue'], 
                           data_provide = True, title = 'Covid-19 Cases - line chart',
                           input_dataset = input_dataset,
                           multi_axis = True)<jupyter_output><empty_output><jupyter_text>Two states had their very first case on March 4th, and the number of cases both leaped to hundred thousand during next two months.   Looking at the cases for all states, most of states had their first case at the begining of March.  `mode` value is used to control the mouseover label  - 'point': present the value of one point  - 'index': present the value of points of all dataset at this x axis value  - 'dataset': present the value of all points of this particular datasets  - 'x': present all values at this x-axis value  - 'y': present all values at this y-axis value<jupyter_code>all_state = get_state_data(data, state_name)
all_dataset = [all_state[s] for s in state_name]
time_series.time_series_Chart('line', state_name, 'cases', date_col = 'date', 
                           data_provide = True, title = 'Covid-19 Cases for all states - line chart',
                           input_dataset = all_dataset,log_axis=True, mode = 'point', intersect = False,
                           width=1500)<jupyter_output><empty_output><jupyter_text>Bar Chart<jupyter_code>state = ['New York', 'New Jersey']
start = '2020-03-01'
state_data = get_state_data(data, state, start = start)
input_dataset = [state_data[s] for s in state]
time_series.time_series_Chart('bar', state, 'cases', date_col = 'date', colors = ['red', 'blue'], 
                               data_provide = True, title = 'Covid-19 Cases - bar chart',
                               input_dataset = input_dataset, 
                               stacked = True)<jupyter_output><empty_output><jupyter_text>From the stacked bar chart, we can notice that the number of Covid-19 cases doubled in New York state.  Daily Increase  in Cases by States  Next, we can look at the daily increase cases for New York and New Jersey.  The following function is defined to extract the number of cases increasing daily.<jupyter_code>def get_daily_increase(data, state, start=None, end=None):
    df = get_state_data(data, state)
    daily_df = {}
    for i in df:
        temp = df[i].set_index('date')[['cases','deaths']].diff().reset_index().fillna(0)
        if start is None:
            start = temp.iloc[0].date
        if end is None:
            end = temp.iloc[-1].date
        daily_df[i] = temp[(temp.date >= start) & (temp.date <= end)]
    return daily_df
state = ['New York', 'New Jersey']
start = '2020-03-01'
daily_data = get_daily_increase(data, state, start=start, end=None)
input_dataset = [daily_data[s] for s in state]
time_series.time_series_Chart('line', state, 'cases', date_col = 'date', colors = ['red', 'blue'], 
                               data_provide = True, title = 'Covid-19 Daily Increase in Cases - line chart',
                               input_dataset = input_dataset)<jupyter_output><empty_output><jupyter_text>From the end of March to the beginning of May, the number of cases increasing by day were more than 4000. On April 7th, the number of daily increase in cases reached 12,000.<jupyter_code>daily_data = get_daily_increase(data, state_name, end=None)
input_dataset = [daily_data[s] for s in state_name]
time_series.time_series_Chart('line', state_name, 'cases', date_col = 'date', 
                               data_provide = True, title = 'Covid-19 Daily Increase in Cases - line chart',
                               input_dataset = input_dataset, intersect = True, width=2000)<jupyter_output><empty_output><jupyter_text>Cases and Deaths - PopulationAn additional dataset about population by states will be used to help demonstrate the information, which can be found [here](https://worldpopulationreview.com/states/).<jupyter_code>pop = pd.read_csv('../data/us-population.csv')
population = pop.set_index('State').to_dict('dict')['Pop'] #key: state; value: population
end_date = data.iloc[-1].date

covid_bubble = bubble.Bubble()

k=0
for s in population:
    df = data[(data.state==s)&(data.date==end_date)]
    x = round(float(df['cases'].values[0]/population[s]*100),2)
    y = round(float(df['deaths'].values[0]/population[s]*100),2)
    r = round(float(np.log(population[s])),2)
    dataset = [{'x':x, 'y': y}]
    covid_bubble.add_dataset(
                            dataset, s,
                            radius = r,
                            backgroundColor = utils.color_rgb(utils.color_name[k], 0.5),
                            borderColor = utils.color_rgb(utils.color_name[k]),
                            hoverBackgroundColor = 'transparent',
                            hoverBorderColor = utils.color_rgb(utils.color_name[k])
                            )
    k+=1
covid_bubble.options.update({'scales':{
    'xAxes':[utils.axes(
                            display = True,
                            scaleLabel = {
                                'display': True,
                                'labelString': 'Positive test/Population'
                                }
                        )],
    'yAxes':[utils.axes(
                            display = True,
                            scaleLabel = {
                                'display': True,
                                'labelString': 'Deaths/Population'
                                }
                        )]
    }})
covid_bubble.set_title("Covid-19 Deaths/Population vs. Cases/Population")
covid_bubble.setup(width=1600)
covid_bubble<jupyter_output><empty_output><jupyter_text>Click to Compare on Date`chart_ipynb` provides several click event functions to allow to do something specifically with the clicked object. More details can refer to [quick reference](https://github.com/AaronWatters/Chart_ipynb/blob/master/notebooks/chart_ipynb%20quick%20reference.ipynb).<jupyter_code>state = ['New York']
start = '2020-06-01'
state_data = get_daily_increase(data, state, start=start, end=None)
input_dataset = [state_data[s] for s in state]
ny_daily = time_series.time_series_Chart('line', state, 'cases', date_col = 'date', colors = ['red'], 
                           data_provide = True, title = 'Covid-19 Daily Increases in Cases - New York',
                           input_dataset = input_dataset)
ny_daily.debugging_display()<jupyter_output><empty_output><jupyter_text>Let's define a function called `higher_daily` to take the information of clicked object as input. The following function extracts date and increase values of New York of the clicked point, and returns a dictionary of dictionary with date as key and state dictionary as value.<jupyter_code>result = {}

def higher_daily(*arguments):
    global result
    day = arguments[0]['label']
    increase_case = arguments[0]['dataValue']
    result[day] = {}
    for i in daily_data:
        val = daily_data[i][daily_data[i].date==day].cases.values[0]
        if val > increase_case:
            result[day][i] = val
    
ny_daily.click_callback(higher_daily)
result<jupyter_output><empty_output><jupyter_text>```{'2020-06-02': {'California': 2438.0, 'Texas': 1885.0, 'Illinois': 1578.0}}```Since the daily increases on June 2nd is highest, we can click on that point to see any other states having higher daily increases. On that day, California, Texas, and Illinois had higher daily increases.  Cases by County —— New York City  We are also interested in specified counties at states, the following example explores into county-level analysis of cases.<jupyter_code>county_url = 'https://raw.githubusercontent.com/nytimes/covid-19-data/master/us-counties.csv'
county_data = pd.read_csv(county_url)
county_data.head(5)
def get_county_data(data, state, county_name=None, start=None, end=None):
    """
    data: pd.DataFrame
    state: a str of state or a list of states
    """
    state_dict = get_state_data(data, state, start=start, end=start)
    state_data = state_dict[state]
    counties = state_data.reset_index(drop=True).groupby('county')
    
    county_dict = {}
    if county_name is None:
        county_name = list(set(county_data.query('state=="%s"' %state).county))
    if isinstance(county_name, str):
        county_name = [county_name]
    for c in county_name:
        idx = counties.groups[c]
        county_dict[c] = state_data.iloc[idx].reset_index(drop=True)
    return county_dict

def get_multi_state_county(data, state, county_name, start=None, end=None):
    '''
    county: a nested dict; key: state; value: county or a list of county
    '''
    state_dict = get_state_data(data, state, start=start, end=start)
    for s in county_name:
        state_df = state_dict[s].reset_index(drop=True).groupby('county')
        county_list = county_name[s]
        county_dict = {}
        if isinstance(county_list, str):
            county_list = [county_list]
        for c in county_list:
            idx = state_df.groups[c]
            county_dict[c] = state_df.iloc[idx].reset_index(drop=True)
        state_dict[s] = county_dict
    return state_dict

def get_daily_county(data, state, county_name,start=None, end=None):
    daily_county_df = {}
    temp_data = get_county_data(data, state, county_name)     
    for c in temp_data:
        temp_county = temp_data[c].set_index('date')[['cases','deaths']].diff().reset_index().fillna(0)
        if start is None:
            start = temp_county.iloc[0].date
        if end is None:
            end = temp_county.iloc[-1].date
        daily_county_df[c] = temp_county[(temp_county.date >= start) & (temp_county.date <= end)]
    return daily_county_df
state = 'New York'
county_name = 'New York City'
df = get_county_data(county_data, state, county_name)
input_dataset = [df[county_name]]
time_series.time_series_Chart('bar', county_name, 'cases', date_col = 'date', colors = ['red'], 
                               data_provide = True, title = 'Covid-19 Cases - New York City',
                               input_dataset = input_dataset)
daily_df = get_daily_county(county_data, state, county_name)
input_dataset = [daily_df[county_name]]
time_series.time_series_Chart('line', county_name, 'cases', date_col = 'date', colors = ['red'], 
                           data_provide = True, title = 'Covid-19 Daily Increase in Cases - New York City',
                           input_dataset = input_dataset)<jupyter_output><empty_output><jupyter_text>Maps and Charts The following example combines the map outline and chart. The map outline can refer to https://github.com/AaronWatters/jp_doodle/blob/master/notebooks/State%20outlines.ipynb. The state area is clickable, after you click on the state, the dropdown list will have clicked state options.<jupyter_code>xml_file = "../data/states.xml"
import xml.etree.ElementTree as ET
tree = ET.parse(xml_file)
root = tree.getroot()
name_to_boundary = {}
allpoints = []
for state in root:
    name = state.attrib["name"]
    #print name
    boundary = []
    for point in state:
        lat = float(point.attrib["lat"])
        lng = float(point.attrib["lng"])
        #pt = [lat, lng]
        pt = [lng, lat]
        boundary.append(pt)
        allpoints.append(pt)
    name_to_boundary[name] = boundary
info_list = []
state_selection = widgets.Dropdown(
            options=['states'],
            value='states',
            description='States:',
            disabled=False,
        )
chart_area = widgets.Box(children=[])
def selection_callback(val):
    s = val['new']
    input_dataset = get_state_data(data, s)[s]
    chart_widget = time_series.time_series_Chart('line', s, 'cases', date_col = 'date', 
                                   data_provide = True, title = 'Covid-19 Cases - %s' %s,
                                   input_dataset = [input_dataset])
    chart_area.children=[chart_widget]
def get_name(info):
    info_list.append(info)
    chart_area = widgets.Box(children = [time_series.time_series_Chart('line', info, 'cases', date_col = 'date', 
                                   data_provide = True, title = 'Covid-19 Cases - %s' %info,
                                   input_dataset = [get_state_data(data, info)[info]])])   
    state_selection.options = list(set(info_list))
    state_selection.value = info_list[-1]
    
state_selection.observe(selection_callback, names='value')
states = dual_canvas.DualCanvasWidget(width=520, height=320)
states.check_jquery()

states.text(text="Longitude", y=-60, x=250, align="center", font="bold 20px Arial",)
states.text(text="Latitude", y=150, x=-50, align="center", degrees=90, font="bold 20px Arial",)
minlng = min(x[0] for x in allpoints)
maxlng = max(x[0] for x in allpoints)
minlat = min(x[1] for x in allpoints)
maxlat = max(x[1] for x in allpoints)


# image underlay
earth_image = 'pics/Earthmap1000x500.jpg'
# local link does not work (as written) in Jupyter Lab
earth_image = 'https://upload.wikimedia.org/wikipedia/commons/a/ac/Earthmap1000x500.jpg'
def latitude_pixel(lat):
    "pixel from top edge = (90 - latitude) / 0,36"
    return (90.0 - lat) / 0.36
def longitude_pixel(lng):
    "pixel from left hand side = (180 + longitude) / 0,36"
    return (180 + lng) / 0.36
sx = longitude_pixel(minlng)
sy = latitude_pixel(maxlat)
sWidth = (maxlng - minlng) / 0.36
sHeight = (maxlat - minlat) / 0.36


states.name_image_url("earth", earth_image)
states.named_image("earth", 0,0,500,300, degrees=0, 
                   sx=sx, sy=sy, sWidth=sWidth, sHeight=sHeight)

# States polygons overlay.
frame = states.frame_region(0,0,500,300,minlng, minlat, maxlng, maxlat)
for name in name_to_boundary:
    points = name_to_boundary[name]
    # add semi-transparent filled polygon for state interior
    frame.polygon(points=points, name=name, color="rgba(100,200,0,0.5)")
    # add unnamed state border
    frame.polygon(points=points, color="#38f", fill=False)

# add reference axes
frame.right_axis(
    min_value= minlat,
    max_value= maxlat,
    max_tick_count= 6,
    axis_origin= dict(x=maxlng+2, y=0),
    tick_line_config= dict(color="#66f"),
    tick_text_config= dict(color="#875"),
)

frame.bottom_axis(
    min_value= minlng,
    max_value= maxlng,
    max_tick_count= 8,
    axis_origin= dict(x=0, y= minlat-2),
    tick_line_config= dict(color="#66f"),
    tick_text_config= dict(color="#875", align="center", valign="center", degrees=0),
    add_end_points= True,
)

#print minlng, minlat, maxlng, maxlat
states.fit()

# Use javascript to add mouse over event handling and a pop-up dialog.

states.js_init("""

// Text info area and JQueryUI dialog
var info = $("<div>info here</div>").appendTo(element);
var dialog = $("<div>dialog text</div>").appendTo(element);
dialog.dialog();

// https://stackoverflow.com/questions/17358622/dialog-box-hide-and-show-jquery
dialog.dialog("open");
// Using dialog.parent().hide() and *.show() avoids scrolling anomalies.
dialog.parent().hide();

var last_name = null
var mouse_move = function(event) {
    var name = event.canvas_name;
    var pos = { my: "left+10 top+10", at: "left bottom", of: event }
    dialog.dialog("option", "position", pos);
    if ((last_name) && ((last_name!=name) || event.type=="mouseout")) {
        // obscure the interior)
        element.change(last_name, {color: "rgba(100,200,0,0.5)"})
        last_name = null;
        dialog.parent().hide();
    }
    if (name) {
        // make the interior transparent
        element.change(name, {color: "rgba(0,0,0,0)"})
        last_name = name;
        dialog.html("<div>"+name+"</div>");
        dialog.parent().show();
    }
    info.html("<div>name="+name+"; last_name="+last_name+"</div>");
}

var mouse_click = function(event) {
    var name = event.canvas_name;
    if (name) {
        //click_state.push(name);
        get_name(name);
    }
}
element.on_canvas_event("mousemove", mouse_move);
element.on_canvas_event("click", mouse_click);
""", get_name=get_name)

display(states, state_selection, chart_area)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1245186136071887, 'fraction_numerical': 0.015725288831835688, 'mean_word_length': 3.2726234003656307, 'pattern_counts': {'":': 0, '<': 64, '<?xml version=': 0, '>': 65, 'https://': 9, 'lorem ipsum': 0, 'www.': 0, 'xml': 4}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '283919', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 6189, 'n_tokens_neox': 5861, 'n_words': 1511, 'path': 'notebooks/Covid19 Examples.ipynb', 'repo_name': 'AaronWatters/Chart_ipynb', 'size': '18696'}