<jupyter_start><jupyter_text><NAME>  Thinkful Capstone 1  April 2019  Mass Shootings in America: 1966-2017 --------------------------------------------------------------------------This dataset was downloaded from kaggle.com. It uses data from public sources including Wikipedia, <NAME>, Stanford, and USA Today.    CONTENTS Geography: United States of AmericaTime period: 1966-2017Unit of analysis: Mass Shooting AttacksVariables: The dataset contains Title, Location, Date, Summary, Fatalities, Injured, Total Victims, Mental Health Issue, Race, Gender, Age, and Lat-Long information.----------------------------------------------------------------There is no single legal definition of a mass shooting. One strict definition used in the past by the federal government says at least four victims must be killed in one incident. Much broader criteria requires at least four victims be injured in one incident, though not necessarily killed. Other definitions land somewhere in the middle. It is also unclear whether the injury or death of the shooter(s) count in the definition. This dataset includes some instances where zero people were killed and three were injured in one incident. Because of this, some people could argue that this data may be biased by including incidents that do not qualify as mass shootings. ---------------------------------------------------------------------ANALYTICAL QUESTIONS1. What is the overall trend in mass shootings?2. Is there a typical shooter profile: age, race, gender, mental illness?3. Is there a correlation between a state's gun laws (loose or strict) and the amount of shootings that have occurred in those states? Do strict gun laws prevent mass shootings compared to states with loose gun laws?<jupyter_code># Import libraries

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
# Import dataset

mass = pd.read_csv(r'Mass Shootings Dataset Version 5.csv')
# General info about the dataset

mass.info()
# Check NaN values in dataframe

mass.isnull().sum()
# Drop NaN values in dataframe

mass.dropna(inplace=True)
# Drop unecessary/irrelevant columns

mass.drop(['S#'],inplace=True,axis=1)
mass.drop(['Employeed (Y/N)'],inplace=True,axis=1)
mass.drop(['Employed at'],inplace=True,axis=1)
# List first three rows of dataset

mass.head(3)
# Split date column and add year column

mass['Year'] = [np.int(i.split('/')[2]) for i in mass['Date']]
# Summary statistics on fatalities, injured, total victims, and avergage age of shooter

mass.loc[:, ['Fatalities', 'Injured', 'Total victims', 'Avg Age']].describe()
mass = mass.sort_values(by=['Year'])
# Overall trend in incidents 1980-2017 in 5-year increments

plt.figure(figsize=(15,6))
mass['Year'].value_counts().sort_index(ascending=True).plot(label='Total Incidents')
mass.groupby('Year').mean()['Injured'].plot()
mass.groupby('Year').mean()['Fatalities'].plot()
mass.groupby('Year').mean()['Total victims'].plot(label='Total Victims')

plt.title('INCIDENTS OVER TIME', fontsize=12)
plt.xlabel('DATE', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.yscale('log')
plt.xlim(1980, 2018)
plt.xticks(list(range(1980, 2020, 5)))
plt.legend()
plt.show()<jupyter_output><empty_output><jupyter_text>This chart shows the overall trend of mass shooting incidents over time. The data seems to show that there has been a significant increase in injuries, fatalities, and total victims from mass shootings since around 2014, while total incidents has remained relatively stable over the last 35 years. This could potentially be because of the person compiling the dataset changing the definition of a mass shooting over time. For example, they could have included incidents in which X amount of injuries occurred since 2010 but excluded those same incidents in prior years.<jupyter_code># Incidents by age

plt.figure(figsize=(12,6))
plt.hist(mass['Avg Age'], bins=20)
plt.xticks(list(range(10, 70, 5)))
plt.xlabel('AGE', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.title('MASS SHOOTINGS BY AGE', fontsize=12)
plt.show()

print("Mean age: ", mass['Avg Age'].mean())
print("Median age: ", mass['Avg Age'].median())
print("Variance: ", mass['Avg Age'].var())
print("Standard deviation: ", mass['Avg Age'].std())
print("Standard error: ", (np.std(mass['Avg Age']) / np.sqrt(len(mass['Avg Age']))))<jupyter_output><empty_output><jupyter_text>This chart shows us that most mass shootings are committed by people between the ages of 15 and 45. The highest rate of incidents are committed by people around the age of 20 and drops off sharply after the age of 45-50. The mean and median age of a typical shooter is 32 years old. Variance is high at 175.<jupyter_code># Boxplot of average ages of shooters

plt.boxplot(mass['Avg Age'])
plt.title('AGE RANGE OF SHOOTER', fontsize=12)
plt.ylabel('AGE', fontsize=12)
plt.show()<jupyter_output><empty_output><jupyter_text>This boxplot shows the age range of mass shooters. The bulk of mass shootings occur by people between the age 20-45. The orange line shows the median age of a shooter at 32. The oldest shooter is about 70 years old, and the youngest looks to be in their early teens.<jupyter_code># Attacks by race top five

# Combine/clean race categories
mass.Race.replace(['white', 'black', 'Some other race', 'unclear'],
                ['White', 'Black', 'Other','Unknown'], inplace=True)
mass.Race.replace(['White American or European American', 'Black American or African American', 'Asian American'], 
               ['White', 'Black', 'Asian'], inplace=True)

race = mass['Race'].value_counts()
race = race.head(5)
plt.figure(figsize=(8,6))
race.plot(kind='bar')
plt.xticks(rotation='vertical')
plt.xlabel('RACE', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.title('MASS SHOOTINGS BY RACE', fontsize=12)
plt.show()

print(race)<jupyter_output><empty_output><jupyter_text>This chart shows that whites and blacks commit the majority of shootings, with whites outnumbering blacks 64 to 21. According to the data whites are 3 times more likely to commit a shooting than blacks. However, this data is not normalized to population size, which could potentially yield different results.<jupyter_code># Attacks by gender

# Combine/clean gender categories
mass["Gender_updated"] = ["Male" if i in ["Male", "M"] else "Female" if i in ["Female"] else "Unspecified" 
                          for i in mass["Gender"]]

gen = mass['Gender_updated'].value_counts()
gen.plot(kind='bar')
plt.xticks(rotation='vertical')
plt.xlabel('GENDER', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.title('MASS SHOOTINGS BY GENDER', fontsize=12)
plt.show()

print(gen)<jupyter_output><empty_output><jupyter_text>From this chart we can see males commit the vast number of shootings compared to females, outnumbering them 105 to 3. Men are 35 times more likely to commit a mass shootings than women. The data is not normalized to population size, but typical demographics for men and women should be fairly evenly skewed.<jupyter_code># Attacks by person with mental health issues

# Combine/clean mental health categories
mass['Mental Health Issues'].replace(['unknown', 'unclear', 'Unclear'], ['Unknown','Unclear', 'Unclear'], inplace=True)
mass['Mental Health Issues'].replace(['Unknown'], ['Unclear'], inplace=True)

mhlth = mass['Mental Health Issues'].value_counts()
mhlth.plot(kind='bar')
plt.xticks(rotation='vertical')
plt.xlabel('MENTAL HEALTH', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.title('MASS SHOOTINGS BY MENTAL HEALTH ISSUES', fontsize=12)
plt.show()

print(mhlth)<jupyter_output><empty_output><jupyter_text>From this chart we can see shootings committed by people with known mental health issues (54) roughly equals those committed by people not known to have prior mental health issues (38). At first glance it is unclear if mental health plays a statistically significant role in mass shootings because there are many shootings by people not know to have prior mental health issues. Let's take a closer look by comparing fatalities and total victims to the mental health of the shooter.-------------------------------------------------------------------------------Null Hypothesis: There is no significant difference in mass shootings between people with mental health issues versus people not known to have mental health issues.<jupyter_code># Separate groups for people with known mental health issues and people with no known mental health issues

mass_gr1 = mass[mass["Mental Health Issues"] == "Yes"]
mass_gr2 = mass[mass["Mental Health Issues"] == "No"]
# Fatalities by people with mental health issues

plt.hist(mass_gr1['Fatalities'])
plt.xlabel('FATALITIES', fontsize=12)
plt.ylabel('FREQUENCY', fontsize=12)
plt.title('FATALITIES BY PEOPLE WITH MENTAL HEALTH ISSUES', fontsize=12)
plt.show()
# Fatalities by people not known to have mental health issues

plt.hist(mass_gr2['Fatalities'])
plt.xlabel('FATALITIES', fontsize=12)
plt.ylabel('FREQUENCY', fontsize=12)
plt.title('FATALITIES BY PEOPLE WITH NO KNOWN MENTAL HEALTH ISSUES', fontsize=12)
plt.show()
# Total victims of people with mental health issues

plt.hist(mass_gr1['Total victims'])
plt.xlabel('VICTIMS', fontsize=12)
plt.ylabel('FREQUENCY', fontsize=12)
plt.title('TOTAL VICTIMS BY PEOPLE WITH MENTAL HEALTH ISSUES', fontsize=12)
plt.show()
# Total victims of people not known to have mental health issues

plt.hist(mass_gr2['Total victims'])
plt.xlabel('VICTIMS', fontsize=12)
plt.ylabel('FREQUENCY', fontsize=12)
plt.title('TOTAL VICTIMS BY PEOPLE WITH NO KNOWN MENTAL HEALTH ISSUES', fontsize=12)
plt.show()<jupyter_output><empty_output><jupyter_text>Since the above four charts show non-normal distributions, we will use the Mann-Whitney U Test to compare results.<jupyter_code>from scipy.stats import mannwhitneyu

s, p = mannwhitneyu(mass_gr1['Fatalities'], mass_gr2['Fatalities'])
print("Fatalities P-value: ", p)

s, p = mannwhitneyu(mass_gr1['Total victims'], mass_gr2['Total victims'])
print("Total victims P-value: ", p)<jupyter_output>Fatalities P-value:  0.0198836653234754
Total victims P-value:  0.019128208502323905<jupyter_text>Because the P-values are low and do not meet the 0.05 threshold, we can reject the initial null hypothesis that the difference is due to random sampling, and conclude instead that the populations are distinct. We can therefore assert that mental health can play a significant role in overall victims of mass shootings.<jupyter_code># Top 5 states of shootings

# Separate cities and states in location column
mass['City'] = mass['Location'].str.rpartition(',')[0]
mass['State'] = mass['Location'].str.rpartition(',')[2]

sts_5 = mass['State'].value_counts()
sts_5 = sts_5.head()
#plt.figure(figsize=(8,6))
sts_5.plot(kind='bar')
plt.xticks(rotation='vertical')
plt.xlabel('STATE', fontsize=12)
plt.ylabel('INCIDENTS', fontsize=12)
plt.title('TOP 5 STATES OF MASS SHOOTINGS', fontsize=12)
plt.show()

print(sts_5)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13211845102505695, 'fraction_numerical': 0.022232346241457857, 'mean_word_length': 5.027457440966502, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '413800', 'license': 'CC-BY-3.0', 'n_tokens_mistral': 3477, 'n_tokens_neox': 3138, 'n_words': 1212, 'path': 'Capstone 1 - Mass Shootings in America.ipynb', 'repo_name': 'pfistermiyagi/pfistermiyagi.github.io', 'size': '10987'}