<jupyter_start><jupyter_text>The Counted (project by The Guardian to count the people killed by police in the US) Why is this necessary?From The Guardian's http://www.theguardian.com/us-news/ng-interactive/2015/jun/01/the-counted-police-killings-us-database"The US government has no comprehensive record of the number of people killed by law enforcement. This lack of basic data has been glaring amid the protests, riots and worldwide debate set in motion by the fatal police shooting of <NAME>, an unarmed 18-year-old, in Ferguson, Missouri, in August 2014."<jupyter_code>!pip install pandas
!pip install matplotlib
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline<jupyter_output>/Users/mercybenzaquen/.virtualenvs/Homework8/lib/python3.5/site-packages/matplotlib/__init__.py:1035: UserWarning: Duplicate key in file "/Users/mercybenzaquen/.matplotlib/matplotlibrc", line #2
  (fname, cnt))<jupyter_text>Open your dataset up using pandas in a Jupyter notebook<jupyter_code>df = pd.read_csv('the-counted-2015.csv', encoding = "ISO-8859-1")<jupyter_output><empty_output><jupyter_text>Do a .head() to get a feel for your data<jupyter_code>df.head()<jupyter_output><empty_output><jupyter_text>Write down 12 questions to ask your data, or 12 things to hunt for in the data  1) How many people were killed by police in 2015?<jupyter_code>df.tail(1) #there is one line per incident, so tail will give us the last incident.<jupyter_output><empty_output><jupyter_text>2)Who was/were the oldest person killed?<jupyter_code>df.sort_values('age', ascending=False).head(10)<jupyter_output><empty_output><jupyter_text>3)Who was/were the youngest person killed?<jupyter_code>df.sort_values('age', ascending=True).head(5)<jupyter_output><empty_output><jupyter_text>4)What was the age average of people killed?<jupyter_code>df['age'].describe()  #I could not get the average :s!

#I thought using describe I would get something like this:
#count    18617.000000
#mean        53.314841
#std         10.679143
#min         25.000000
#25%         45.400000
#50%         53.000000
#75%         60.500000
#max         98.100000
#Name: age, dtype: float64<jupyter_output><empty_output><jupyter_text>5)What was the state with more killings by police in 2015?<jupyter_code>df['state'].value_counts()<jupyter_output><empty_output><jupyter_text>6)What was the city with more killings by police in 2015?<jupyter_code>df['city'].value_counts()<jupyter_output><empty_output><jupyter_text>7)List all the incidents in Los Angeles<jupyter_code>los_angeles = df['city'] == 'Los Angeles'
#df['complete_date'] = df['day'], df['year']
df[los_angeles]

#I wanted to add a new column to order the incidents chronologically (month, day, year) but I got an error saying, 
#ValueError: Length of values does not match length of index<jupyter_output><empty_output><jupyter_text>8) What was the month with more police killings in 2015?<jupyter_code>df['month'].value_counts()<jupyter_output><empty_output><jupyter_text>9) What was the day with more police killings in July?<jupyter_code>df['July'] = df['month'] == 'July'
df['July'].value_counts()
july_count = df.groupby('month')['day'].value_counts()
pd.DataFrame(july_count)

#I tried to do df.groupby('July')['day'].value_counts() but it did not work.
#Here I am getting the results but I am not able to see July.<jupyter_output><empty_output><jupyter_text>10) How are these killings distributed by race?<jupyter_code>df['raceethnicity'].value_counts()
#this results do not align with those in the Guardian's website. ???<jupyter_output><empty_output><jupyter_text>11) And by gender?<jupyter_code>df['gender'].value_counts()<jupyter_output><empty_output><jupyter_text>12) How many of the people killed where carrying a firearm?<jupyter_code>df['armed'].value_counts()
df.head(20)<jupyter_output><empty_output><jupyter_text>13) Which was the law enforcement agency with more killings?<jupyter_code>df['lawenforcementagency'].value_counts()<jupyter_output><empty_output><jupyter_text>13) How many people were killed in custody?<jupyter_code>df['classification'].value_counts()<jupyter_output><empty_output><jupyter_text>14) List the people killed who where male, Hispanic/Latino and armed with a knife?<jupyter_code>male = df['gender'] == 'Male'
latino = df['raceethnicity'] == 'Hispanic/Latino'
knife = df['armed'] == 'Knife'
df[male&latino&knife]

#How can I count them now? It tells me it is not possible?
df['gender'].value_counts()
#doesn't look right!
plt.style.use('ggplot')
df['age'].value_counts().hist()
#we had to create a new list since histograms need floats and we had strings (unknown)

age2 = []
for point in df['age']:
    if point != 'Unknown':
        age2.append(float(point))
    else:
        age2.append(0)

df['age_2'] = age2

df['age_2'].hist()

df['age'].sort_values()
        
#we still have unknown values plotted
no_unknowns = df.drop(df.index[[147, 1072, 1066, 88]])

no_unknowns['age'].sort_values()

age2 = []
for point in no_unknowns['age']:
    age2.append(float(point))

no_unknowns['age2'] = age2

no_unknowns['age2'].hist()
df['state'].value_counts().hist()

df['state'].value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.12463824040131198, 'fraction_numerical': 0.033957167663515336, 'mean_word_length': 6.311706629055007, 'pattern_counts': {'":': 0, '<': 73, '<?xml version=': 0, '>': 73, 'https://': 0, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '18', 'dir': 'jupyter-structured-clean-dedup', 'id': '386039', 'license': 'MIT', 'n_tokens_mistral': 1915, 'n_tokens_neox': 1761, 'n_words': 472, 'path': 'foundations_hw/08/Homework8_benzaquen_police_killings.ipynb', 'repo_name': 'mercybenzaquen/foundations-homework', 'size': '5190'}