<jupyter_start><jupyter_text>Relative Crime IndexSince the data is not comprehensive(there are approximate 950,000 crimes for year 2019 and 2020), we cannot directly compare numbers between this two dataframe. Thus, we use relative crime index instead, which is RCI = case_num / mean(case_num).<jupyter_code>mean20 = crime20['case_num'].mean()
crime20['RCI'] = crime20['case_num'] / mean20
crime20['RCI']
mean19 = crime19['case_num'].mean()
crime19['RCI'] = crime19['case_num'] / mean19
crime19['RCI']

rci19 = crime19.plot(column='RCI', legend=True, cmap='gist_heat_r')
rci20 = crime20.plot(column='RCI', legend=True, cmap='gist_heat_r')<jupyter_output><empty_output><jupyter_text>Weighted Difference of RCIvariation * weight. For districts with a few crime incidents, a small increasing in total number of crimes will lead to dramatic increasing on RCI, and for ones with lots crime incidents, a small difference on RCI actually means huge increasing or decreasing. Therefore, we multiply the difference and mean of number of crime incidents of two years to get our weighted differences of RCI.<jupyter_code>RCI_diff = crime20[['geometry']]
RCI_diff['diff'] = (crime20['RCI'] - crime19['RCI']) * (crime19['RCI'] + crime20['RCI']) / 2
norm = mpl.colors.Normalize(vmin=-1.,vmax=1.)
RCI_diff.plot(column='diff', legend=True, cmap='RdBu_r', norm=norm)<jupyter_output><empty_output><jupyter_text>It seems that Manhattan' weighted CRI decreased significantly, this might due to the "shut down"  Covid-19<jupyter_code>covid = gpd.read_file('data/covid_geo.geojson')
covid['RCI'] = crime20['RCI']
covid
covid = covid[covid.pop_deno != 0]
covid['pos_rate'] = covid['case_num'] / covid['pop_deno']
covid['death_rate'] = covid['death_num'] / covid['pop_deno']
covid['dt_rate'] = covid['death_num'] / covid['case_num']
covid['RCI_rate'] = covid['RCI'] / covid['pop_deno']
covid['RCI_diff'] = RCI_diff['diff'] 
covid
covid_plot = covid.plot(column='case_num', legend=True, cmap='gist_heat_r', figsize=(12, 8))
%matplotlib inline
plt.scatter(covid['RCI_rate'], covid['pos_rate'])
plt.title("Relationship between RCI per Person and Positive Rate")
plt.xlabel("RCI per Person \n Correlation = -0.1152")
plt.ylabel("Positive Rate")
covid['RCI_rate'].corr(covid['pos_rate'])<jupyter_output><empty_output><jupyter_text>Death Rate<jupyter_code>%matplotlib inline
plt.scatter(covid['RCI_rate'], covid['dt_rate'])
plt.title("Relationship between RCI per Person and Death per Positive")
plt.xlabel("RCI per Person \n Correlation = 0.1596")
plt.ylabel("Death per Positive")
covid['RCI_rate'].corr(covid['dt_rate'])
%matplotlib inline
plt.scatter(covid['RCI_diff'], covid['pos_rate'])
plt.title("Relationship between Weighted Difference on RCI and Positive Rate")
plt.xlabel("Weighted Difference on RCI \n Correlation = 0.1418")
plt.ylabel("Positive Rate")
covid['RCI_diff'].corr(covid['pos_rate'])
%matplotlib inline
plt.scatter(covid['RCI_diff'], covid['dt_rate'])
plt.title("Relationship between Weighted Difference on RCI and Death per Positive")
plt.xlabel("Weighted Difference on RCI \n Correlation = 0.08961")
plt.ylabel("Death per Positive")
covid['RCI_diff'].corr(covid['dt_rate'])<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14544885830466062, 'fraction_numerical': 0.02752580544260244, 'mean_word_length': 7.666666666666667, 'pattern_counts': {'":': 0, '<': 17, '<?xml version=': 0, '>': 17, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '438481', 'license': 'MIT', 'n_tokens_mistral': 1207, 'n_tokens_neox': 1103, 'n_words': 272, 'path': 'overview_part2_RCI.ipynb', 'repo_name': 'chengzegang/Relationship-between-Covid-19-and-Unlawful-Incidents-in-New-York-City', 'size': '3197'}