<jupyter_start><jupyter_text>Full AnalysisThis file runs all analyses associated with project group 13's dataset. To clarify, the research questions are as follows:Rhys: How does substance type correlate to event type? (i.e., how do opioids and stimulants compare regarding event types?)Shalini:(1)Which regions of Canada have experienced the most opioid related deaths in 2020? (2)Has the opioid crisis been getting better or worse in canada?<jupyter_code># importing libraries and functions
#to ignore the pandas library error
import warnings
warnings.simplefilter(action='ignore', category=FutureWarning)

#libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt

#paths
import os
import sys
sys.path.append('..')

#our function & profile reports
from pandas_profiling import ProfileReport
from scripts import project_functions_v2 as pf<jupyter_output><empty_output><jupyter_text>Rhys' AnalysisSubstance use and event types:<jupyter_code>#load dataset and check head
df_s = pf.load_process_rdata('../data/raw/SubstanceHarmsData.csv')
#print(df_s.head())
#Comparing substance vs. TOTAL hospitalizations and deaths
#note: for substance, 1 = Opioids, 2 = stimulants
# EMS data is unavailable for stimulants

plt.figure()
source_counts = sns.countplot(data=df_s,y="Source",hue="Substance",palette="colorblind")
source_counts.set_title("Deaths, Hospitalizations, and EMS by Substance")
source_counts.set_xlabel("Total Occurrences")
source_counts.set_ylabel("Source of Data")
source_counts.legend(title="Substance",labels=["Opioids","Stimulants"])

plt.show()
#comparing all event types by substance
#combining event types (total/accidental/intentional deaths/hospitalisations)
df_e = df_s
df_e["Grouped Events"] = df_e["Type of Event"]
df_e["Grouped Events"] = df_e["Type of Event"].replace(to_replace=["Total apparent opioid toxicity deaths","Total apparent stimulant toxicity deaths"],value="Total apparent substance deaths",inplace=True)
df_e["Grouped Events"] = df_e["Type of Event"].replace(to_replace=["Accidental apparent opioid toxicity deaths","Accidental apparent stimulant toxicity deaths"],value="Accidental apparent substance deaths",inplace=True)
df_e["Grouped Events"] = df_e["Type of Event"].replace(to_replace=["Total opioid-related poisoning hospitalizations","Total stimulant-related poisoning hospitalizations"],value="Total substance-related poisoning hospitalisations",inplace=True)
df_e["Grouped Events"] = df_e["Type of Event"].replace(to_replace=["Accidental opioid-related poisoning hospitalizations","Accidental stimulant-related poisoning hospitalizations"],value="Accidental substance-related poisoning hospitalisations",inplace=True)
df_e["Grouped Events"] = df_e["Type of Event"].replace(to_replace=["Intentional opioid-related poisoning hospitalizations","Intentional stimulant-related poisoning hospitalizations"],value="Intentional substance-related poisoning hospitalisations",inplace=True)

print(df_e["Type of Event"].unique())

# plotting
plt.figure()
ev_plt = sns.countplot(data=df_e,y="Type of Event",hue="Substance",palette="muted")
ev_plt.set_title("Types of Event by Substance")
ev_plt.set_xlabel("Total Occurrences")
ev_plt.set_ylabel("Type of Event")
ev_plt.legend(title="Substance",labels=["Opioids","Stimulants"])

plt.show()<jupyter_output>['Total apparent substance deaths' 'Accidental apparent substance deaths'
 'Suicide apparent opioid toxicity deaths'
 'Total substance-related poisoning hospitalisations'
 'Accidental substance-related poisoning hospitalisations'
 'Intentional substance-related poisoning hospitalisations'
 'EMS responses to suspected opioid-related overdoses']<jupyter_text>Exporting DataWe can export the data to a .csv file (to be used in Tableau later)<jupyter_code>df_e.to_csv("../data/processed/Events_Substances.csv",sep=",")<jupyter_output><empty_output><jupyter_text>Comments Regarding Substance Use and Event TypesTwo groupings were used to provide information regarding the correlation between substances and medical events. First, we see summaries of deaths, hospitalizations, and EMS responses by the type of substance. From this graph (which has no information for stimulant EMS responses), we can see that despite comparable levels of hospitalizations, opioids account for more than double the amount of deaths recorded.\\From the second chart, we can see the consistency regarding substance use and death. However, from this chart we can see that the vast majority of accidental apparent substance deaths occur with opioid usage. From this, we can tailor any future analyses and perhaps explore the role of opioids in accidental poisonings more closely.   Shalini's Analysis<jupyter_code># import all required module
import pandas as pd
import seaborn as sns
import numpy as np
import os
import matplotlib.pyplot as plt
from pandas_profiling import ProfileReport

# import loading of data method
from scripts import project_functions_v2

# Load and proces data
df = project_functions_v2.load_and_process("https://health-infobase.canada.ca/src/doc/SubstanceHarmsData.csv")
#display(df.head())

# convert "Value" column into integers and assign it to a new column
df['Value_int'] = df['Value'].astype(int)

# print new bank lines
print("\n\n")<jupyter_output><jupyter_text>**Research Question 1**: Which regions of Canada have experienced the most opioid related deaths in 2020?<jupyter_code>#graph 1
df1 = df.groupby(["Region","Year_Quarter"]).filter(lambda x: (x["Year_Quarter"] == '2020').any())
df1 = df1.reset_index()
#display(df1.head())

sns.set_theme(style="whitegrid", font_scale=1.5 )
sns.barplot(data = df1,x = 'Region', y = 'Value_int', color = 'red')
plt.xticks(fontsize = 15, rotation=45, ha='right')
plt.yticks(fontsize = 15)
plt.xlabel('Regions', size=12)
plt.ylabel('Total apparent Opioid toxicity deaths', size=12)
plt.title('Total Opioid Related Deaths in Canada for 2020', size=20, pad=40)
plt.show()

# print new bank lines
print("\n\n")<jupyter_output><empty_output><jupyter_text>**RQ1 Conclusion**:- A bar graph was created to visualize the relationship between region and death count. Based on bargraph 1, Ontario seems to have experienced the most opioid related deaths in 2020 followed by British Columbia and Alberta respectively. The high percentage of deaths primarily occurring in those 3 regions indicates that government efforts need to be amped up drastically in those areas. Overall, this information and visualization aids in providing us with a nation-wide picture of the opioid crisis in 2020.  **Research Question 2**: Has the opioid crisis been getting better or worse in canada?<jupyter_code># Graph 2
df2 = df.groupby('Year_Quarter').Value_int.sum()
df2 = df2.reset_index()
#display(df2.head())

sns.set_theme(style="darkgrid", font_scale=1.5 )
sns.lineplot(data = df2,x = 'Year_Quarter', y = 'Value_int', color = 'blue')
plt.xticks(fontsize = 15, rotation=45, ha='right')
plt.yticks(fontsize = 15)
plt.xlabel('Year', size=12)
plt.ylabel('Opioid related death count', size=12)
plt.title('Total Opioid related deaths in Canada : 2016 - 2020', size=20, pad=40)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10068330776739645, 'fraction_numerical': 0.012550550829730861, 'mean_word_length': 7.094808126410835, 'pattern_counts': {'":': 0, '<': 23, '<?xml version=': 0, '>': 23, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '6', 'dir': 'jupyter-structured-clean-dedup', 'id': '175547', 'license': 'MIT', 'n_tokens_mistral': 2217, 'n_tokens_neox': 2020, 'n_words': 698, 'path': 'analysis/All_Analyses.ipynb', 'repo_name': 'data301-2021-summer2/project-group13-project', 'size': '7171'}