<jupyter_start><jupyter_text>Challenging Tickets The majority of drivers (58.6%) who were given a ticket did not challenge it<jupyter_code>taxi_tickets_hearing = taxi_tickets.groupby(['Hearing Dispo'])['Ticket Number'].count()
taxi_tickets_hearing.plot(kind='pie',autopct='%1.1f%%', fontsize=16)<jupyter_output><empty_output><jupyter_text>Out of those who challenged it though, 72.7% were found not liable for the ticket<jupyter_code>taxi_challenged = taxi_tickets[taxi_tickets['Hearing Dispo'] != 'Did Not Challenge']
taxi_challenge_hearing = taxi_challenged.groupby(['Hearing Dispo'])['Ticket Number'].count()
taxi_challenge_hearing.plot(kind='pie',autopct='%1.1f%%',fontsize=16)<jupyter_output><empty_output><jupyter_text>Residential and Street Parking  For residential and business street parking violations alone, a higher percentage (68.5% vs. 58.6% for all tickets) of drivers did not challenge their tickets<jupyter_code>taxi_bus_res = taxi_tickets[taxi_tickets['Violation Code'].isin(['0964170A','0964170B'])]
taxi_st_hearing = taxi_bus_res.groupby(['Hearing Dispo'])['Ticket Number'].count()
taxi_st_hearing.plot(kind='pie',autopct='%1.1f%%', fontsize=16)<jupyter_output><empty_output><jupyter_text>Additionally, for these two types of ticket violations, a lower percentage of drivers were successful at a hearing, with 52.2% of drivers being deemed Not Liable, vs. 72.7% of drivers for tickets overall<jupyter_code>taxi_st_challenged = taxi_bus_res[taxi_bus_res['Hearing Dispo'] != 'Did Not Challenge']
taxi_st_chal_hearing = taxi_st_challenged.groupby(['Hearing Dispo'])['Ticket Number'].count()
taxi_st_chal_hearing.plot(kind='pie',autopct='%1.1f%%', fontsize=16)<jupyter_output><empty_output><jupyter_text>Challenges By Category<jupyter_code># Getting the top 25 categories to look at challenge rates to ignore much less common tickets
top_categories = taxi_tickets.groupby(['Violation Description'])['Ticket Number'].count()
top_categories = top_categories.sort_values(ascending=False)
top_25_categories = top_categories.index[:25].tolist()
top_category_tickets = taxi_tickets[taxi_tickets['Violation Description'].isin(top_25_categories)]
category_hearings = top_category_tickets.groupby(['Violation Description', 'Hearing Dispo'])['Ticket Number'].count()
viol_hearing_pct = pd.DataFrame(category_hearings).reset_index()
viol_hearing_pct['Percent'] = viol_hearing_pct.groupby('Violation Description')['Ticket Number'].apply(lambda x: x.astype(float)/x.sum() * 100)
viol_hearing_pct.head()
not_challenged_cat = viol_hearing_pct[viol_hearing_pct['Hearing Dispo'] == 'Did Not Challenge']
not_challenged_cat = not_challenged_cat[['Violation Description', 'Percent']].sort_values(['Percent'], ascending=False)
not_challenged_cat.head()<jupyter_output><empty_output><jupyter_text>Out of the 25 most common types of tickets received by taxis, residential and business street parking are 8th and 11th in the amount of tickets that go unchallenged, with 69.9% and 64.0% respectively going unchallenged<jupyter_code>not_challenged_cat.plot(x='Violation Description', y='Percent', kind='bar')<jupyter_output><empty_output><jupyter_text>Additionally, part of the reason for these tickets being relatively not likely to be challenged is that business street parking tickets are the most likely for a driver to be held liable for if challenged, and residential parking is 6th most likely.  The average outcome (including tickets Not Challenged) for business street parking tickets was Liable 16.2% of the time, and it was 11.5% for residential street parking tickets<jupyter_code>liable_cat = viol_hearing_pct[viol_hearing_pct['Hearing Dispo'] == 'Liable']
liable_cat = liable_cat[['Violation Description', 'Percent']].sort_values(['Percent'], ascending=False)
liable_cat[:10]
not_challenged_cat.plot(x='Violation Description', y='Percent', kind='bar')<jupyter_output><empty_output><jupyter_text>Looking at percentages of outcomes by category Both business and residential street parking tickets were less likely than many of the other top 25 categories for drivers to result in a Not Liable hearing. Business street parking tickets led to Not Liable 17.5% of the time, while residential were Not Liable 14.2% of the time<jupyter_code>not_liable_cat = viol_hearing_pct[viol_hearing_pct['Hearing Dispo'] == 'Not Liable']
not_liable_cat = not_liable_cat[['Violation Description', 'Percent']].sort_values(['Percent'])
not_liable_cat[:15]<jupyter_output><empty_output><jupyter_text>Challenge Rate for Taxis OverallWhile most people don't receive too many tickets a year, taxi drivers are driving much more by necessity, and so it's worth looking at, out of the tickets an individual driver receives, how many are they likely to challenge Notes* Even looking at the percentages of tickets that drivers challenge out of the tickets they receive, they still challenge at a higher rate than the general public (though slightly lower)* Interestingly, when looking at only drivers with more than one ticket, the challenge rate increases substantially* Additionally, when removing the outliers where the percent of challenged tickets is 100%, challenge rate drops significantly    * Could be interesting to see which companies the drivers who are challenging most often are with (if any)<jupyter_code>driver_total_tix = pd.DataFrame(taxi_tickets.groupby(['License Plate Number'])['Ticket Number'].count()).reset_index()
driver_total_tix.head()
driver_not_challenged = taxi_tickets[taxi_tickets['Hearing Dispo'] == 'Did Not Challenge']
driver_not_challenged_df = pd.DataFrame(driver_not_challenged.groupby(['License Plate Number'])['Ticket Number'].count()).reset_index()
driver_challenged = taxi_tickets[taxi_tickets['Hearing Dispo'].isin(['Liable', 'Not Liable', 'Denied', 'Continued', 'Stricken'])]
driver_challenged_df = pd.DataFrame(driver_challenged.groupby(['License Plate Number'])['Ticket Number'].count()).reset_index()

driver_merged = driver_total_tix.merge(driver_not_challenged_df, on='License Plate Number', how='left')
driver_merged = driver_merged.rename(columns={'Ticket Number_x': 'Total Tickets', 'Ticket Number_y': 'Tickets Not Challenged'})
driver_merged = driver_merged.merge(driver_challenged_df, on='License Plate Number', how='left')
driver_merged = driver_merged.rename(columns={'Ticket Number': 'Tickets Challenged'})
driver_merged = driver_merged.fillna(value=0.0)
driver_merged['Percent Challenged'] = (driver_merged['Tickets Challenged'] / driver_merged['Total Tickets']) * 100
driver_merged.head()
driver_top_challenge = driver_merged.sort_values(by='Percent Challenged', ascending=False)
driver_top_challenge[:10]
mean_challenge = driver_merged['Percent Challenged'].mean()
print('The mean percentage of challenging tickets by individual taxis was: {}%'.format(mean_challenge))
median_challenge = driver_merged['Percent Challenged'].median()
print('The median percentage of challenging tickets by individual taxis was: {}%'.format(median_challenge))
# To identify how much the outliers impacted the challenge rate, looking at the mean challenge rate for taxis with 
# more than one ticket as well as mean for taxis where the challenge rate is below 100%
driver_gt_one_mean = driver_merged[driver_merged['Total Tickets'] > 1]['Percent Challenged'].mean()
driver_gt_one_median = driver_merged[driver_merged['Total Tickets'] > 1]['Percent Challenged'].median()
print('The mean percentage of challenging tickets by individual taxis with more than one ticket was: {}%'.format(driver_gt_one_mean))
print('The median percentage of challenging tickets by individual taxis with more than one ticket was: {}%'.format(driver_gt_one_median))
driver_lt_all_mean = driver_merged[driver_merged['Percent Challenged'] < 100]['Percent Challenged'].mean()
driver_lt_all_median = driver_merged[driver_merged['Percent Challenged'] < 100]['Percent Challenged'].median()
print('The mean percentage of challenging tickets by individual taxis where it is less than 100%: {}%'.format(driver_lt_all_mean))
print('The median percentage of challenging tickets by individual taxis where it is less than 100%: {}%'.format(driver_lt_all_median))<jupyter_output>The mean percentage of challenging tickets by individual taxis where it is less than 100%: 18.0334816739%
The median percentage of challenging tickets by individual taxis where it is less than 100%: 0.0%
{'fraction_non_alphanumeric': 0.10411680229775012, 'fraction_numerical': 0.016275730014360938, 'mean_word_length': 8.40044994375703, 'pattern_counts': {'":': 0, '<': 38, '<?xml version=': 0, '>': 38, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '280058', 'license': 'MIT', 'n_tokens_mistral': 2640, 'n_tokens_neox': 2405, 'n_words': 736, 'path': 'Challenging Tickets.ipynb', 'repo_name': 'pjsier/taxi-parking-tickets', 'size': '8356'}