<jupyter_start><jupyter_text>This shows that there is an obvious negaative correlation between the number of crimes and the median Income level. Communities with a higher income level have a much lower rate of crimes per population. We also noticed that there is one big outlier, where the medIncome is listed at the highest, but there is still a high rate of crimes, which could skew the data.<jupyter_code>print(cleaned_df['ViolentCrimesPerPop'].mean())
print(cleaned_df['ViolentCrimesPerPop'].median())<jupyter_output>0.23797893681043028
0.15<jupyter_text>The mean and median ViolentCrimesPerPop will help us to later see what should be classified as a low, medium and high levels of crime. We can now see that it is probably unusualy for the violentcrimesperpop rate to be above .5, and those with a higher rate should be classified as such.<jupyter_code># encode state labels to range from [0, n-1]
le = preprocessing.LabelEncoder()
le.fit(cleaned_df['state'])

cleaned_df['state'] = le.transform(cleaned_df['state'])

grouped_state = cleaned_df.groupby('state').mean()
grouped_state['ViolentCrimesPerPop'].plot.bar()<jupyter_output><empty_output><jupyter_text>This made us realize, that the numbers relating to states are off and some numbers are skipped, resulting in 56 states. We will have to go back a check if any states are missing/reassign states to their correct numbers as to not create confusion in the future.<jupyter_code>size = cleaned_df['racePctWhite']*100
fig, ax = plt.subplots()
cleaned_df.plot.scatter(x="numbUrban", y = "ViolentCrimesPerPop", s = size, c= 'medIncome',
                        colormap= 'jet', ax=ax)<jupyter_output><empty_output><jupyter_text>This graph has many different attributes to compare, as we saw before that as medIcome increases, as does the rate of violent crimes, which we see since all red and orange dots are at the bottom. As well we can see on the x axis that there are not many communities with a high number of people living in urban areas, but for the ones that do have a higher percentage, they also tend to have an income on the lower part of the scale and a higher rate of violent crimes. Lastly, the size of each dot represent the percent of population that is caucasian. While it is difficult to see exactly, since theere are so many data points, it seems as the datapoints get closer to the lower left corner, the dots tend to be larger, concluding that the percent of caucasians is larger when the income is higher and the violent crimes are lower.<jupyter_code>fig, ax = plt.subplots()
cleaned_df.plot.scatter(x='PctNotHSGrad', y='PctPopUnderPov', c = 'ViolentCrimesPerPop',
                        colormap= 'jet', ax=ax)<jupyter_output><empty_output><jupyter_text>From this we can see that there is a strong positive correlation between the percent of the population under the poverty level and the percent that have not graduated highschool. As one increases, as does the other. As well, we see that as each increases, there is a much greater number of non-blue data point, meaning that the rate of violent crimes as well increases.<jupyter_code>import seaborn as sns
sns.distplot(cleaned_df.ViolentCrimesPerPop)
fig, ax = plt.subplots()
size = cleaned_df['racepctblack']*100
cleaned_df.plot.scatter(x='PctIlleg', y='PctKids2Par', s=size, c='ViolentCrimesPerPop',
                        colormap='Reds', ax=ax)
plt.ylabel("Pct kids in family housing w/ two parents")
plt.xlabel("Pct kids born to never married")<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07167235494880546, 'fraction_numerical': 0.00938566552901024, 'mean_word_length': 5.063793103448276, 'pattern_counts': {'":': 0, '<': 20, '<?xml version=': 0, '>': 20, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '458636', 'license': 'MIT', 'n_tokens_mistral': 1035, 'n_tokens_neox': 1002, 'n_words': 452, 'path': 'crime_term_project/Data Visualization.ipynb', 'repo_name': 'caticardelle/eece2300_termproject', 'size': '3516'}