<jupyter_start><jupyter_text>Assignment Data Description- covid data of daily cummulative cases of India as reported from January 2020 to 8th August 2020- Source: https://www.kaggle.com/sudalairajkumar/covid19-in-india Conduct below Insight investigation1. Find which state has highest mean of cummulative confirmed cases since reported from Jan 2020- Plot line graph plotting means of top 10 States having highest daily confirmed cases2. Which state has highest Death Rate for the month of June, July & Aug - Plot bar graph of Death Rates for the top 5 states3. Explore Trend in Confirmed Cases for the state of Maharashtra - Plot line graph with x axis as Date column and y axis as daily confirmed cases. - such a graph is also called as Time Series Plot Below key steps to be adopted to solve above Questions- Load Data --> Clean data / Data munging --> Grouping of Data by State --> Exploration using plots- update required code where everywhere '?' is mentioned in the code block  Load Packages<jupyter_code>import pandas as pd # for cleaning and loading data from csv file
import numpy as np
from matplotlib import pyplot as plt # package for plotting graphs
import datetime
import seaborn as sns; sns.set(color_codes=True)
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Load data<jupyter_code>df = ?
df.head() # Preview first 5 rows of dataframe
# Convert Date column which is a string into datetime object
df["Date"] = pd.to_datetime(df["Date"], format = "%d/%m/%y")
df.head()<jupyter_output><empty_output><jupyter_text>Explore more about datetime conversion of column in pandas here:https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.to_datetime.html?highlight=datetimepandas.to_datetime  Cleaning of data- The dataset consists of cummulative values, aim is to create columns with daily reported deaths and confirmed cases.- Below method is helper function to create column consisting of daily cases reported from Cummulative freq column<jupyter_code>ex = np.unique(df['State/UnionTerritory'])
ex<jupyter_output><empty_output><jupyter_text>From above unique values of states it is clear that Telangana is represented in multiple ways. We will change each occurrence of Telangna state with standard spelling<jupyter_code>def clean_stateName(stateName):
    if stateName == 'Telangana***':
        stateName = 'Telangana'
    elif stateName == 'Telengana':
        stateName = 'Telangana'
    elif stateName == 'Telengana***':
        stateName = 'Telangana'
    
    return stateName<jupyter_output><empty_output><jupyter_text>- Apply method is used to apply either user defined or builtin function across every cell of dataframe- Commonly lambda function is used to apply method across each cell- A lambda function is a small anonymous function.- A lambda function can take any number of arguments, but can only have one expression.<jupyter_code>df["State/UnionTerritory"] = df["State/UnionTerritory"].apply(lambda x: clean_stateName(x))
np.unique(df["State/UnionTerritory"]) # to identify all unique values in a column of dataframe or array<jupyter_output><empty_output><jupyter_text>Helper function to find daily confirmed cases from cumulative values<jupyter_code>def daily_cases(dframe, stateColumn,dateColumn, cummColumn):
    # Sort column containing state and then by date in ascending order
    dframe.sort_values(by = [stateColumn, dateColumn], inplace = True)
    newColName = 'daily_' + cummColumn
    dframe[newColName] = dframe[cummColumn].diff() # diff is pandas method to caclucate difference between consecutive values
#     print(dframe.tail())
    '''
    Below line uses shift method of pandas to compare consecutive state names and if they are not different
    as shown by using ! symbol then create list of boolean, True for if they are different else False
    ''' 
    mask = dframe[stateColumn] != dframe[stateColumn].shift(1)
    dframe[newColName][mask] = np.nan # where value of mask =True the cell value will be replaced by NaN
    dframe[newColName] = dframe[newColName].apply(lambda x: 0 if x < 0 else x) # replace negative values by 0
#     dframe.drop('diffs',axis=1, inplace = True)
    
    return dframe
    
df_new = daily_cases(dframe= df, stateColumn= 'State/UnionTerritory',dateColumn= 'Date', cummColumn= 'Confirmed')
# Preview data 
df_new[df_new["State/UnionTerritory"]=="Maharashtra"].tail(n=5)<jupyter_output><empty_output><jupyter_text>Q1. Find which state has highest mean of cummulative confirmed cases since reported from Jan 2020<jupyter_code># Hint : Groupby state names to find their means for confirmed cases
df_group = ?
df_group = df_group.sort_values(ascending= False)[0:10]
df_group
df_group.index
ax = sns.lineplot(x= ? , y= df_group.values) # df_group.values - extract only the values of a column
plt.scatter(x= ? , y= ? , c = 'r')
ax.figure.set_figwidth(12)
ax.figure.set_figheight(4)
ax.set_ylabel("Mean of Daily Confirmed Cases")<jupyter_output><empty_output><jupyter_text>Q2. Which state has highest Death Rate for the month of June, July & Aug<jupyter_code># Hint - explore how a datetime column of dataframe can be filtered using specific months as a list 
df_months = df_new['Date'].apply(lambda x: x.month in ?) # this will create boolean basis comparison of months from a list
df_final = df_new[df_months] # Filtered dataframe consisting of data from June, July & Aug
df_final.tail()
df_final['death_rate'] = df_final['Deaths'] / df_final['Confirmed'] *100
df_final.tail()
df_groups_deaths = ?
top_10_deathrates = ?
fig, ax = plt.subplots()
fig.set_figwidth(15)
fig.set_figheight(6)
ax.bar(x = ?, height =  ?)
ax.set_xlabel('States')
ax.set_ylabel('Death Rates %')
ax.set_title('Top 10 States with Highest Death Rate since June 2020')
for i, v in enumerate(top_10_deathrates.values):
    ax.text(i, v, s = ("%.2f" % v), color='blue', fontweight='bold', fontsize = 12) # %.2f will print decimals upto 2 places
plt.xticks(rotation=45) # this line will rotate the x axis label in 45 degrees to make it more readable<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08662588857662423, 'fraction_numerical': 0.010745577781451479, 'mean_word_length': 5.123481781376518, 'pattern_counts': {'":': 0, '<': 34, '<?xml version=': 0, '>': 36, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '353231', 'license': 'MIT', 'n_tokens_mistral': 1874, 'n_tokens_neox': 1770, 'n_words': 675, 'path': 'covid_data_analysis-Assignment.ipynb', 'repo_name': 'rahulkumbhar8888/DataScience', 'size': '6049'}