<jupyter_start><jupyter_text>______ Data Analysis with Python Project - 1 - Traffic Police Stops   Does the ``gender`` of a driver have an impact on police behavior during a traffic stop? **In this chapter**, you will explore that question while practicing filtering, grouping, method chaining, Boolean math, string methods, and more! ***  Examining traffic violations Before comparing the violations being committed by each gender, you should examine the ``violations`` committed by all drivers to get a baseline understanding of the data.In this exercise, you'll count the unique values in the ``violation`` column, and then separately express those counts as proportions. > Before starting your work in this section **repeat the steps which you did in the previos chapter for preparing the data.** Continue to this chapter based on where you were in the end of the previous chapter.<jupyter_code>import pandas as pd
df = pd.read_csv("police.csv", nrows = 50000)
df1 = df.dropna(axis=1, how= "all")
df1.drop("state", axis=1, inplace = True)
df1.dropna(subset = ["driver_gender"], inplace = True)
df1["is_arrested"].dtypes
df1["is_arrested"] = df1["is_arrested"].astype("bool")
combined = df1["stop_date"].str.cat(df1["stop_time"], sep = " ")
df1["stop_datetime"] = pd.to_datetime(combined)
df1.set_index("stop_datetime", inplace = True)<jupyter_output><empty_output><jupyter_text>**INSTRUCTIONS***   Count the unique values in the ``violation`` column, to see what violations are being committed by all drivers.*   Express the violation counts as proportions of the total.<jupyter_code>df1["violation"].unique()
df1["violation"].nunique()
df1["violation"].value_counts(normalize=True)
df1["violation"].value_counts()
df1["violation"].value_counts()/sum(df1["violation"].value_counts())
df1.groupby("violation").size()
df1.groupby("violation").size()/sum(df1.groupby("violation").size())<jupyter_output><empty_output><jupyter_text>***  Comparing violations by gender The question we're trying to answer is whether male and female drivers tend to commit different types of traffic violations.You'll first create a ``DataFrame`` for each gender, and then analyze the ``violations`` in each ``DataFrame`` separately. **INSTRUCTIONS***   Create a ``DataFrame``, female, that only contains rows in which ``driver_gender`` is ``'F'``.*   Create a ``DataFrame``, male, that only contains rows in which ``driver_gender`` is ``'M'``.*   Count the ``violations`` committed by female drivers and express them as proportions.*   Count the violations committed by male drivers and express them as proportions.<jupyter_code>f_df = df1[df1["driver_gender"] == "F"]
m_df = df1[df1["driver_gender"] == "M"]
f_df["violation"].value_counts(normalize = True)
m_df["violation"].value_counts(normalize = True)<jupyter_output><empty_output><jupyter_text>***  Comparing speeding outcomes by gender When a driver is pulled over for speeding, many people believe that gender has an impact on whether the driver will receive a ticket or a warning. Can you find evidence of this in the dataset?First, you'll create two ``DataFrames`` of drivers who were stopped for ``speeding``: one containing ***females*** and the other containing ***males***.Then, for each **gender**, you'll use the ``stop_outcome`` column to calculate what percentage of stops resulted in a ``"Citation"`` (meaning a ticket) versus a ``"Warning"``. **INSTRUCTIONS***   Create a ``DataFrame``, ``female_and_speeding``, that only includes female drivers who were stopped for speeding.*   Create a ``DataFrame``, ``male_and_speeding``, that only includes male drivers who were stopped for speeding.*   Count the **stop outcomes** for the female drivers and express them as proportions.*   Count the **stop outcomes** for the male drivers and express them as proportions.<jupyter_code>m_df_speeding = df1[(df1["driver_gender"] == "M") & (df1["violation"] == "Speeding")]
f_df_speeding = df1[(df1["driver_gender"] == "F") & (df1["violation"] == "Speeding")]
m_df_speeding["stop_outcome"].value_counts(normalize = True)
f_df_speeding["stop_outcome"].value_counts(normalize = True)<jupyter_output><empty_output><jupyter_text>***  Calculating the search rate During a traffic stop, the police officer sometimes conducts a search of the vehicle. In this exercise, you'll calculate the percentage of all stops that result in a vehicle search, also known as the **search rate**. **INSTRUCTIONS***   Check the data type of ``search_conducted`` to confirm that it's a ``Boolean Series``.*   Calculate the search rate by counting the ``Series`` values and expressing them as proportions.*   Calculate the search rate by taking the mean of the ``Series``. (It should match the proportion of ``True`` values calculated above.)<jupyter_code>df1["search_conducted"].dtype
df1["search_conducted"].value_counts(normalize = True)
df1["search_conducted"].mean()<jupyter_output><empty_output><jupyter_text>***  Comparing search rates by gender You'll compare the rates at which **female** and **male** drivers are searched during a traffic stop. Remember that the vehicle search rate across all stops is about **3.8%**.First, you'll filter the ``DataFrame`` by gender and calculate the search rate for each group separately. Then, you'll perform the same calculation for both genders at once using a ``.groupby()``. **INSTRUCTIONS 1/3***   Filter the ``DataFrame`` to only include **female** drivers, and then calculate the search rate by taking the mean of ``search_conducted``.<jupyter_code>f_df["search_conducted"].mean()<jupyter_output><empty_output><jupyter_text>**INSTRUCTIONS 2/3***   Filter the ``DataFrame`` to only include **male** drivers, and then repeat the search rate calculation.<jupyter_code>m_df["search_conducted"].mean()<jupyter_output><empty_output><jupyter_text>**INSTRUCTIONS 3/3***   Group by driver gender to calculate the search rate for both groups simultaneously. (It should match the previous results.)<jupyter_code>df1.groupby("driver_gender").search_conducted.mean()<jupyter_output><empty_output><jupyter_text>***  Adding a second factor to the analysis Even though the search rate for males is much higher than for females, it's possible that the difference is mostly due to a second factor.For example, you might hypothesize that the search rate varies by violation type, and the difference in search rate between males and females is because they tend to commit different violations.You can test this hypothesis by examining the search rate for each combination of gender and violation. If the hypothesis was true, you would find that males and females are searched at about the same rate for each violation. Find out below if that's the case! **INSTRUCTIONS 1/2***   Use a ``.groupby()`` to calculate the search rate for each combination of gender and violation. Are males and females searched at about the same rate for each violation?<jupyter_code>df1.groupby(["driver_gender", "violation"]).search_conducted.mean()<jupyter_output><empty_output><jupyter_text>**INSTRUCTIONS 2/2***   Reverse the ordering to group by violation before gender. The results may be easier to compare when presented this way.<jupyter_code>df1.groupby(["violation", "driver_gender"]).search_conducted.mean()<jupyter_output><empty_output><jupyter_text>***  Counting protective frisks During a vehicle search, the police officer may pat down the driver to check if they have a weapon. This is known as a ``"protective frisk."``You'll first check to see how many times "Protective Frisk" was the only search type. Then, you'll use a string method to locate all instances in which the driver was frisked. **INSTRUCTIONS***   Count the ``search_type`` values to see how many times ``"Protective Frisk"`` was the only search type.*   Create a new column, frisk, that is ``True`` if ``search_type`` contains the string ``"Protective Frisk"`` and ``False`` otherwise.*   Check the data type of frisk to confirm that it's a ``Boolean Series``.*   Take the sum of frisk to count the total number of frisks.<jupyter_code>df1[df1["search_type"] == "Protective Frisk"]["search_type"].count()
df1["search_type"].value_counts()
df1["frisk"] = df1["search_type"] == "Protective Frisk"
# df1['frisk'] = df1["search_type"].str.contains('Protective Frisk', na=False)
df1["frisk"].value_counts()
df1["frisk"].dtype
sum(df1["frisk"])
df1["frisk"].sum()<jupyter_output><empty_output><jupyter_text>***  Comparing frisk rates by gender You'll compare the rates at which female and male drivers are frisked during a search. Are males frisked more often than females, perhaps because police officers consider them to be higher risk?Before doing any calculations, it's important to filter the ``DataFrame`` to only include the relevant subset of data, namely stops in which a search was conducted. **INSTRUCTIONS***   Create a ``DataFrame``, searched, that only contains rows in which ``search_conducted`` is ``True``.*   Take the mean of the frisk column to find out what percentage of searches included a frisk.*   Calculate the frisk rate for each gender using a ``.groupby()``.<jupyter_code>df_scd = df1[df1["search_conducted"] == True]
df_scd["frisk"].mean()
df_scd.groupby("driver_gender").frisk.mean()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11192819292743593, 'fraction_numerical': 0.0073537363469233265, 'mean_word_length': 6.236306729264475, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '12', 'dir': 'jupyter-structured-clean-dedup', 'id': '298415', 'license': 'MIT', 'n_tokens_mistral': 2644, 'n_tokens_neox': 2535, 'n_words': 1059, 'path': '01-DAwithPython_TrafficPoliceStops_project/02-exploring-the-relationship-between-gender-and-policing.ipynb', 'repo_name': 'ssezerer/Projects', 'size': '9247'}