<jupyter_start><jupyter_text>Part 4 - Visualizing geo-data Choose a crimetype with a spatial pattern that changes over timeWe have chosen the crimetype `Driving under the influence` on an hourly basis, in the hours 0-24, where we see a very prominent spatial pattern. In the hours [4, 14] it seems low and stable, but the later into the evening and night it gets the more the frequency rises. We expect this will also be evident in the heat maps shown in the coming exercises.A bar plot of the data can be seen in the plot below.<jupyter_code>plt.figure(figsize=(12, 5))
df["Hour"] = pd.to_datetime(df.Time).dt.hour
df[df.Category == "DRIVING UNDER THE INFLUENCE"]\
    .Hour\
    .value_counts()\
    .sort_index()\
    .plot.bar(title='Total number of offenses in "Driving under the influence" per hour from 2003 to 2018 San Fransisco')
plt.xlabel("Hour")
plt.ylabel("Number of total offenses")
plt.show()<jupyter_output><empty_output><jupyter_text>Create a heatmap<jupyter_code>df = pd.read_csv("../data/Police_Department_Incident_Reports__Historical_2003_to_May_2018.csv")
df["Hour"] = pd.to_datetime(df.Time).dt.hour
# Ensuring that long (Y) and lat (X) are floats
if df.X.dtype != np.dtype(np.float64):
    df.X = df.X.astype(np.float64)
if df.Y.dtype != np.dtype(np.float64):
    df.Y = df.Y.astype(np.float64)
print(f"Total data size: {df[df.Category == 'DRIVING UNDER THE INFLUENCE'].shape[0]}")
heat_df = df[df.Category == "DRIVING UNDER THE INFLUENCE"]
heat_df = heat_df[["X", "Y"]]
heat_df = heat_df.dropna(axis=0, subset=["X", "Y"])

# Creating list of lists
heat_data = [[row.Y, row.X] for _, row in heat_df.iterrows()]<jupyter_output>Total data size: 5652<jupyter_text>Parameters of heatmapsWe have chosen to show two different heatmaps. The first heatmaps purpose is to show where an offense of driving under the influence is most likely to occur. This is done by lowering the minimum oppacity, so we can see where the highest number of occurences happen. This is especially visible if the map is zoomed out, why `zoom_start=12`. The second map has a of purpose to show if there are special areas where a lot of crimes happen. This is done by lowering the radius and increasing the min_opacity. With this we hope to find a cluster of crimes in the same spot. One of these are highlighted with a marker in the second plot. Interestingly we also see in this plot, that a lot of crimes are accumilated in traffic crosses. Our suspicion is, that an easy way to write down where an offense happened is to write the names of the crossing streets, and this is why the crimes are reported as they are.<jupyter_code># Where are crimes most likely to happen
map = folium.Map(location=[37.76, -122.4192],
                 tiles="Stamen Toner",
                 zoom_start = 12)
plugins.HeatMap(heat_data, 
                min_opacity=.2
                ).add_to(map)
map
# Where are crimes most likely to happen
map = folium.Map(location=[37.76, -122.4192],
                 tiles="Stamen Toner",
                 zoom_start = 13)
plugins.HeatMap(heat_data, radius=25).add_to(map)
folium.Marker([37.775343, -122.403398], 
              popup="Area with high amount of concentrated crimes"
              ).add_to(map)
map<jupyter_output><empty_output><jupyter_text>Create time heatmapWe have chosen to show a heatmap of the activity of DRIVING UNDER THE INFLUENCE crimes on a 24 hour basis, that is there are 24 time steps.<jupyter_code># Where are crimes most likely to happen
map = folium.Map(location=[37.76, -122.4192],
                 tiles="Stamen Toner",
                 zoom_start = 12)

heat_df = df[df.Category == "DRIVING UNDER THE INFLUENCE"][["X", "Y", "Hour"]]
heat_data = [[
    [row.Y, row.X] for _, row in heat_df[heat_df.Hour == hour].iterrows()]
    for hour in range(0, 23)
]
hm = plugins.HeatMapWithTime(heat_data, auto_play=True, max_opacity=.8)
hm.add_to(map)
map<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09892911779704233, 'fraction_numerical': 0.02779194288628251, 'mean_word_length': 3.9160401002506267, 'pattern_counts': {'":': 0, '<': 16, '<?xml version=': 0, '>': 16, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '4', 'dir': 'jupyter-structured-clean-dedup', 'id': '575344', 'license': 'MIT', 'n_tokens_mistral': 1342, 'n_tokens_neox': 1231, 'n_words': 468, 'path': 'exercises/week4_2.ipynb', 'repo_name': 'Moesen/socialdata2022', 'size': '3922'}