<jupyter_start><jupyter_text>Data Analysis: Out of the following categories, which contributes the most towards solved / unsolved cases:- **Race of the victim**- **Gender of the victim**- **Age of the victim**- **Time of year**  Race of victim vs solved / unsolved cases<jupyter_code>df1 = race_analysis(df)
df1
plt.xticks(rotation=45)
sns.barplot(data=df1,y='Count',x='Victim Race', hue = 'Crime Solved')<jupyter_output><empty_output><jupyter_text>Observation:From this, we can see that although there is a larger number of victims of the white race than any other in this dataset, the number of unsolved cases for the black race is larger, suggesting race as a possible factor in solving a case. Because the numbers for Asian / Pacific Islanders as well as Native American / Alaskan Natives are too small, let's take a closer look at the solved vs unsolved cases for these.<jupyter_code>sns.barplot(data=df1.loc[lambda x: x['Victim Race'] != 'Black'].loc[lambda x: x['Victim Race'] != 'White'],y='Count',x='Victim Race', hue = 'Crime Solved')<jupyter_output><empty_output><jupyter_text>Further Observations:From this, we can closer get an idea of the distribution between solved and unsolved cases for these groups. In order to take a closer look, we should calculate the percentage of cases solved for each race to see if there is a trend. Some hand calculations result in the following values for percentage solved:- *Asian / Pacific Islander Percentage*: **70.5 % solved**- *Black*: **66.3% solved**- *Native American / Alaska Native*: **79.9% solved**- *White*: **74.1% solved**From this, it appears that most races have a percentage of solved cases within ± 10% of each other, which although seeming reasonable becomes odd when considering the total number of victims per race.Total Victims: - *Asian / Pacific Islander Percentage*: **9,888**- *Black*: **299,858**- *Native American / Alaska Native*: **4,566**- *White*: **317,367**Due to the number of cases, the percentile of solved cases per race becomes less impressive, as there is still an 80% solve rate for the white race, but other races with much less victims still have a lower percent of cases solved.  Gender of victim vs solved / unsolved cases<jupyter_code>df2 = gender_analysis(df)
df2
sns.barplot(data=df2,y='Count',x='Victim Sex', hue = 'Crime Solved')<jupyter_output><empty_output><jupyter_text>Observation:This graph depicts the distribution of solved vs unsolved cases for the sex of the victim. From this we can see that there is in general less victims which are females. In order to take a closer more precise look, we can again look at the percentages for each gender.The hand calculated percentages are shown below:- *Female*: **77.1% Solved** - *Male*: **68.4% Solved**Again comparing this against the total number of victims for each gender can give us some context for these percentages.- *Female*: **141,993** - *Male*: **489,686**Although there are *29%* less female victims than male victims, the number of solved cases is only approximately 9% higher, suggesting the gender is possibly a factor in whether or not a case gets solved. An interesting investigation to do from this would be to analyze whether or not cases involving victims that are females of minority races are solved less.     Analysis of both race and gender<jupyter_code>df3 = racegender_analysis(df)
df3
g = sns.FacetGrid(df3, col="Victim Race",row='Crime Solved',hue = 'Crime Solved' ,height=5, aspect=1)
g.map(sns.barplot, "Victim Sex", "Count", order=["Male", "Female"])
#g.set(ylim=(0, 100000))<jupyter_output><empty_output><jupyter_text>Initial observations:As we know, there are more male victims than female overall within this dataset, and although cases with female victims are solved 9% more, investigating the ratio for more minority groups can give us a better idea of the potential correlation between gender, race, and if the case gets solved. This view of the dataset can provide us with a couple predictions:- In relation to solved cases involving a female victim, there is approximately 20% to 50% less unsolved cases involving female victims of that race- The percentage of unsolved cases is the highest with cases involving black male victims- Cases involving minority males are solved less, despite cases involving white males being solved the most<jupyter_code>g = sns.FacetGrid(data =df3.loc[lambda x: x['Victim Race']!='White'].loc[lambda x: x['Victim Race']!='Black'], col="Victim Race",row='Crime Solved',hue = 'Crime Solved' ,height=5, aspect=1.5)
g.map(sns.barplot, "Victim Sex", "Count", order=["Male", "Female"])
#g.set(ylim=(0, 100000))<jupyter_output><empty_output><jupyter_text>Closer look:On average, it appears that throughout the minority races, approximately 75% less cases involving female victims go unsolved compared to cases involving male victims. This contradicts our original estimate of 20-50%. Although this data is not entirely conclusive - it would be reasonable to say that the chances of a murder being solved when the victim is female are higher than if the victim were to be male, regardless of race. Exploring Further:- The percentage of unsolved cases is the highest with cases involving black male victims- Cases involving minority males are solved less, despite cases involving white males being solved the mostThese previous observations which were made above suggest that murders involving males of minority races are less likely to be solved, we will investigate this.<jupyter_code>g = sns.FacetGrid(data =df3.loc[lambda x: x['Victim Race']!='Asian/Pacific Islander'].loc[lambda x: x['Victim Race']!='Native American/Alaska Native'], col="Victim Race",row='Crime Solved',hue = 'Crime Solved' ,height=5, aspect=1.5)
g.map(sns.barplot, "Victim Sex", "Count", order=["Male", "Female"])<jupyter_output><empty_output><jupyter_text>Observation:From this, we can clearly see that even though there are fewer cases in total involving black males, the number of unsolved cases is nearly 25% larger when compared to cases involving white males. Although this evidence is not entirely conclusive, this suggests that race is potentially a factor determining whether a case is solved when the victim is male (Although, race may also be a factor when the victim is female as well, however, our previous observation seems to suggest otherwise)   Year of Crime vs Solved & Unsolved Cases<jupyter_code>dfYear = year_analysis(df)
dfYear<jupyter_output>/opt/miniconda3/lib/python3.8/site-packages/pandas/core/ops/array_ops.py:253: FutureWarning: elementwise comparison failed; returning scalar instead, but in the future will perform elementwise comparison
  res_values = method(rvalues)<jupyter_text>Let's begin by looking at a Stacked rea grGaph of solved vs unsolved by yearThis will give us some ratio to look for and see any patterns by year. We can begin to look at the relationship between year and if the case was solved.<jupyter_code>dfYear2 = dfYear.pivot_table(index='Year', columns='Crime Solved', values='Count')
plt.style.use('seaborn')
dfYear2.plot.area()
plt.ylabel('Number of Homicides')
plt.title('Homicides by Solved and Unsolved from 1980 - 2014',fontsize=17)<jupyter_output><empty_output><jupyter_text>Observations:It seems as if a great percentage of crimes are solved more than not but we can spot out some observations from this heat map such as:- A period from 1990 - 1996 where murder was very numerous, solved or not- During the 1980s murder was solved more often than not- Post 1996 while murder rates dropped so did the amount of solved cases, the ratio between solved and unsolved cases dips down to a closer 1 in 4 not solved approximately. from around a 1 in 6.  Let's confirm our suspictions with a bar graph to see if crime rates fell yet crime stayed unsolved post 1996<jupyter_code>fig_dims = (18, 3)
fig, ax = plt.subplots(figsize=fig_dims)
sns.barplot(data=dfYear,y='Count',x='Year', hue = 'Crime Solved', ax=ax)<jupyter_output><empty_output><jupyter_text>As we predicted! Crime rate fell and the Orange solved bar fell with it but the unsolved bar stayed relatively consistentWe can see from this that indeed...It seems that the ratio of solved to unsolved cases pre-1996 was consistent but after the crime wave of 1988 to 1996 it began to take on a more 25/75 approach.We can use this to show that year is a factor in the ability to solve a case. It seemed it was more successful to solve a case in 1996 and earlier while post 1996 solving a homicide case was more difficult which may be due to a variety of factors.   Age of victim vs solved and unsolved cases<jupyter_code>#cleaning outlier, victim age cannot be 998
tooold = df[ df["Victim Age"] == 998 ].index
df.drop(tooold , inplace=True)

#load data
df2=victim_age_analysis(df)

df2<jupyter_output><empty_output><jupyter_text>Initial observation of Victim Age trend<jupyter_code>vicage = df["Victim Age"]
fig_dims = (15, 7)
fig, ax = plt.subplots(figsize=fig_dims)
vicageplot = sns.distplot(vicage, ax=ax).set_title('Victim Age distribution')
plt.show()<jupyter_output><empty_output><jupyter_text>From this graph, we can see that there is a high frequency of murders of infants, this frequency decreases throughout childhood years but peaks again around adulthood (early to mid 20s) then the frequence of murders experiences a gradual decrease as age increasesThere is a slight peek at the age of 99. This could mean that seniors are more likely targeted, or this could be a unique way for agencies to log information (unknown age would be logged as 99 by default) Comparing solved vs unsolved cases for victim age<jupyter_code>dimension = (20, 7)
fig, ax = plt.subplots(figsize=dimension)
sns.kdeplot(data=df, x="Victim Age", hue="Crime Solved", multiple="stack").set_title('Victim Age for Solved vs Unsolved cases')
plt.show()<jupyter_output><empty_output><jupyter_text>Ages with the most unsolved cases range from late teenage years to mid twenties, this gradually decreases as age increases. There are slight peaks for babies and elderly people. This mostly looks like it correlates to the to the number of solved crimes based on victim age, however the peak for unsolved cases of infants is a lot lower. While this does visualize the disparity between solved vs unsolved cases for victim age, looking at the percentage of ssolved cases might be more helpful<jupyter_code>unsolved = df2.loc[df2['Crime Solved'] == 'No']
solved = df2.loc[df2['Crime Solved'] == 'Yes']

perc = unsolved.merge(solved, on ="Victim Age")
perc["perc"] =ratio.Count_y/(ratio.Count_y+ratio.Count_x)

perc
dimension = (30, 7)
fig, ax = plt.subplots(figsize=dimension)
sns.barplot(data=perc,x='Victim Age',y='perc', color="blue", saturation=0.2).set_title('Percentage of Solved cases by Victim Age')
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08305586385497596, 'fraction_numerical': 0.018867924528301886, 'mean_word_length': 5.601343101343101, 'pattern_counts': {'":': 0, '<': 52, '<?xml version=': 0, '>': 52, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '332997', 'license': 'MIT', 'n_tokens_mistral': 3198, 'n_tokens_neox': 2976, 'n_words': 1424, 'path': 'analysis/submitted/.ipynb_checkpoints/Milestone 2-checkpoint.ipynb', 'repo_name': 'data301-2020-winter1/course-project-group_6005', 'size': '10812'}