<jupyter_start><jupyter_text>**What Department Services are most frequently called?**<jupyter_code># Adding a feature called "service" from the 'title' field to indicate if Fire, EMS, or Police were involved
df['title'].head()
# add a column called 'service' with the first token on the left side of the colon in title
df['service'] = df['title'].apply(lambda title: title.split(':')[0])
# changeup 'Traffic' to indicate police services
df['service'] = np.where(df.title.str.contains('Traffic'),'Police',df.service)
df.head(3)<jupyter_output><empty_output><jupyter_text>EMS Services are the most frequently called, then Police, then Fire<jupyter_code>df['service'].value_counts()
# plot the services use
sns.countplot('service',data=df)<jupyter_output><empty_output><jupyter_text>**Doing time analysis**<jupyter_code># What dtype is timestamp?
df.info()
type(df.timeStamp[0])
time = pd.to_datetime(df['timeStamp'][0])
# Convert the timeStamp entries to be
df['timeStamp'] = pd.to_datetime(df['timeStamp'])
df['Date'] = df['timeStamp'].apply(lambda time: time.date())
#df['Date'] = pd.to_datetime(df['timeStamp'])
# Now create features for Hour of Day, Month of Year and Day of Week to simplify Analysis
df['Hour'] = df['timeStamp'].apply(lambda t: t.hour)
df['Month'] = df['timeStamp'].apply(lambda t: t.month)
df['Day of Week'] = df['timeStamp'].apply(lambda t: t.dayofweek)
df.head(2)
# Make the Day Of Week more readable with DayNames
days = {0:'Mon',1:'Tue',2:'Wed',3:'Thu',4:'Fri',5:'Sat',6:'Sun'}
df['Day of Week'] = df['Day of Week'].map(days)<jupyter_output><empty_output><jupyter_text>No Calls on Tuesdays. Also the weekends are weirdly quiet.<jupyter_code># Now show Emergency calls by Service and Day of Week
sns.countplot(x=df['Day of Week'],hue=df['service'], order=['Mon','Tues','Wed','Thu','Fri','Sat','Sun'])
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.) # move legend outside plot<jupyter_output><empty_output><jupyter_text>Looking at months<jupyter_code>byMonth = df.groupby('Month').count()
byMonth.reset_index()
# use this to draw a line for the calls per township. This fills in the missing data in the viz
byMonth['twp'].plot()
sns.countplot(x='Month',data=df,palette='viridis')
# showing calls in zipcode by Month with linear regression. Reset the index on byMonth for the plot
# Notice on the right how the Error shade grows when the line moves past December
sns.lmplot(x='Month',y='zip',data=byMonth.reset_index(), markers=['^'])
# Counting by date at latitude
plt.figure(figsize=(10,4))
df.groupby('Date').count()['twp'].plot(kind='line')
#plt.tight_layout()
# Tidying up Month Names
# remap the month
# Make the map to show names
monthmap = {1:'Jan',2:'Feb',3:'Mar',4:'Apr',5:'May',6:'Jun',7:'Jul',8:'Aug',9:'Sep',10:'Oct',11:'Nov',12:'Dec'}
df['MonthName'] = df['Month'].map(monthmap)<jupyter_output><empty_output><jupyter_text>No data for Oct, Nov, Dec and January<jupyter_code>sns.countplot(x=df['MonthName'],hue=df['service'], order=['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec'])
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.) # move legend outside plot<jupyter_output><empty_output><jupyter_text>Remapping the order to show the gap more clearly.Police get many more calls in February than the summer, which is counterintuitive.<jupyter_code>sns.countplot(x=df['MonthName'],hue=df['service'], order=['Dec','Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov'])
plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.) # move legend outside plot<jupyter_output><empty_output><jupyter_text>**Breaking out a timeseries view by service**<jupyter_code>services=['Police','EMS','Fire']
for s in services:
    df[df['service']==s].groupby('Date').count()['twp'].plot(label=s,figsize=(15,5))
    plt.title(f'Responses By Department')
    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)<jupyter_output><empty_output><jupyter_text>**Adding Heatmaps**<jupyter_code># using .unstack() to pivot the data into a matrix
df.groupby(by=['Day of Week','Hour']).count()['service'].unstack()
dayHour = df.groupby(by=['Day of Week','Hour']).count()['service'].unstack()<jupyter_output><empty_output><jupyter_text>**Easier to see the high counts are working hours on all weekdays, with a slight drop on Sunday**<jupyter_code>plt.figure(figsize=(10,4))
sns.heatmap(dayHour,cmap='coolwarm')<jupyter_output><empty_output><jupyter_text>**Adding a clustermap view for specific time/day correlations**<jupyter_code>sns.clustermap(dayHour,linecolor='black',linewidths=.10,cmap='coolwarm')<jupyter_output><empty_output><jupyter_text>**How about by Month?**<jupyter_code>monthHour = df.groupby(by=['Day of Week','Month']).count()['service'].unstack()<jupyter_output><empty_output><jupyter_text>**Saturdays in January are weirdly active, Sundays in December the least active**<jupyter_code>sns.heatmap(monthHour,cmap='coolwarm')<jupyter_output><empty_output><jupyter_text>**This is a little easier to see with the clustermap**<jupyter_code>sns.clustermap(monthHour,linecolor='black',linewidths=.10,cmap='coolwarm')<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.17763926762758084, 'fraction_numerical': 0.012465913517724971, 'mean_word_length': 8.913127413127413, 'pattern_counts': {'":': 0, '<': 57, '<?xml version=': 0, '>': 57, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '14', 'dir': 'jupyter-structured-clean-dedup', 'id': '642984', 'license': 'BSD-2-Clause', 'n_tokens_mistral': 1886, 'n_tokens_neox': 1790, 'n_words': 392, 'path': '911 Call Analysis v2.ipynb', 'repo_name': 'jsutch/Data_Analysis_and_Visualization_With_Python', 'size': '5134'}