<jupyter_start><jupyter_text>Dave's Data Science Project, Spring 2019  Obisety is a huge problem in America and leads to diseases that kill Americans. Many people estimate that the percentage of people obese is over 30% in America. America is above the world average and is first of the major world powers. Many factors have been linked to obesity including portion sizes, wealth, laziness, more inactive jobs, cost of healthy food and many more. For my project I wanted to put some of these factors to the test against obesity. Instead of doing the whole country I wanted to focus on one part or region to see if any specific factors effected obesity more since that seems to be less common in obesity research. I grew up in New Mexico and saw that many of the states around it had a wide range of obesity from being well above to well below the national average. Those two factors made me decide to focus on the south west region. Specifically the counties in Oklahoma, Kansas, Nevada, Wyoming, Colorado and Utah. I decided to exclude California do to its massive size and used Texas as a boarder to the more south east cities. 	I collected my data using policy map through Tufts University. Policy map is a website that provides different sets of public data mostly from censuses in America to provide data for scientific and educational purposes. Policy map is a business it makes data interactive and people subscribe to it like universities and research labs.       The three factors I will be comparing to percent of the population that is obese are percent of adults that engage in binge drinking, percent of adults reporting to be physically inactive and rate of fast food restaurants. Drinking, physical inactivity and fast food are all well known as being healthy. (ADD MORE LATER)  Files IncludedThe included packages that help with creating and visualizing my analysis.<jupyter_code>from io_hw import io_hw
import pandas as pd
import matplotlib.pyplot as pyplot
import matplotlib.cm as cm
import numpy as np
import math<jupyter_output><empty_output><jupyter_text>Initial Data ParsingHere, the io_hw.p file is used, and the desired fields are then extracted from the dataframe.<jupyter_code>def read_data():
	df, head_df = io_hw('data.csv')
	obese = df.iloc[:, 3].tolist()
	drink = df.iloc[:, 4].tolist()
	inactive = df.iloc[:, 5].tolist()
	fastfood = df.iloc[:, 6].tolist()
	return obese, drink, inactive, fastfood
tempdf, temphead = io_hw('data.csv')
tempdf<jupyter_output><empty_output><jupyter_text>Data Cleaning (removing NaN values):NaN values can hurt the accuracy of my analysis, as well as cause problems in code, so I remove them. However, I want to use as much data as possible during each peice of analysis, and so I rerun this clean code for each y vector.<jupyter_code>def clean(x_init, y_init):
	ziplist = zip(x_init, y_init)
	x_clean = [x for (x, y) in ziplist if not math.isnan(y)]
	ziplist = zip(x_init, y_init)
	y_clean = [y for (x, y) in ziplist if not math.isnan(y)]
	return x_clean, y_clean<jupyter_output><empty_output><jupyter_text>Creating Correlation Graphs:These graphs display the correlation between obesity and the desired y variable.<jupyter_code>def plot(x_init, y_init, name):
    x, y = clean(x_init, y_init)
    pyplot.title(name + " vs Obesity by County in the Midwest")
    pyplot.xlabel("Obesity (%)")
    pyplot.ylabel(name + "(%)")
    curr_plot = pyplot.scatter(x, y)<jupyter_output><empty_output><jupyter_text>Finding Variable Correlation: This function assists in figuring out the correlation coefficients between the two inputted variables. For this use, x_init will always be the obesity vector.<jupyter_code>def correlation(x_init, y_init):
	x, y = clean(x_init, y_init)
	x_np = np.array(x)
	y_np = np.array(y)
	return np.corrcoef(x_np, y_np)[0][1]<jupyter_output><empty_output><jupyter_text>Plotting Figures:<jupyter_code>obese, drink, inactive, fastfood = read_data()

plot(obese, drink, "Binge Drinking")
plot(obese, inactive, "Inactivity")
plot(obese, fastfood, "Fast Food Proximity")<jupyter_output><empty_output><jupyter_text>CorrelationThis piece of code finds each of the test factors' correlation with obesity, and then prints the factor with the highest magnitude of correlation.<jupyter_code>drink_corr = correlation(obese, drink)
inactive_corr = correlation(obese, inactive)
fastfood_corr = correlation(obese, fastfood)
highest_corr = max([drink_corr, inactive_corr, fastfood_corr], key=abs)
zip_corr = [("Drink Correlation", drink_corr), 
		("Inactivity Correlation", inactive_corr), 
		("Fast Food Correlation", fastfood_corr)]
highest_corr_name = [x for (x, y) in zip_corr if highest_corr==y]
print(highest_corr_name[0] + ": " + str(highest_corr))<jupyter_output>Inactivity Correlation: 0.9113725043321461<jupyter_text>As we can see from the (talk about correlation)<jupyter_code>drink_corr
inactive_corr
fastfood_corr<jupyter_output><empty_output><jupyter_text>Plotting All Correlation using Heat MapThe pyplot package is used again here to map my four initial variables against eachother in a heat map. This is useful for visualizing the correlation of each of the test variables with eachother, as well as with obesity.<jupyter_code>def plot_all(obese, drink, inactive, fastfood):
	ziplist = zip(obese, drink, inactive, fastfood)
	o_clean = [x for (x, y, z, w) in ziplist if not math.isnan(w)]
	ziplist = zip(obese, drink, inactive, fastfood)
	d_clean = [y for (x, y, z, w) in ziplist if not math.isnan(w)]
	ziplist = zip(obese, drink, inactive, fastfood)
	i_clean = [z for (x, y, z, w) in ziplist if not math.isnan(w)]
	ziplist = zip(obese, drink, inactive, fastfood)
	f_clean = [w for (x, y, z, w) in ziplist if not math.isnan(w)]

	df_dict = {"Obesity": obese, "Binge Drinking": drink, "Inactivity": inactive, "Fast Food Proximity": fastfood}
	df = pd.DataFrame(df_dict)
	fig = pyplot.figure()
	ax = fig.add_subplot(111)
	cmap = cm.get_cmap('jet', 30)
	cax = ax.imshow(df.corr(), interpolation = "nearest", cmap = cmap)
	ax.grid(True)
	pyplot.title('Correlation Between Test Variables and Obesity')
	labels = ["", "Obesity", "", "Binge Drinking", "", "Inactivity", "", "Fast Food", ""]
	ax.set_xticklabels(labels)
	ax.set_yticklabels(labels)
	fig.colorbar(cax)
	pyplot.show()
plot_all(obese, drink, inactive, fastfood)<jupyter_output><empty_output><jupyter_text>Adding in New DataBelow, new data is added in. The location stays as the same as desired, but new factors are added to see if they contribute to obesity.<jupyter_code>new_df = pd.read_csv('merge dataset.csv')
new_df<jupyter_output><empty_output><jupyter_text>Plotting New DataSimilarly to above, the new data is inputted into the plot function and the correlation between each new variable and obesity is shown.<jupyter_code>education = new_df.iloc[:, 3].tolist()
income = new_df.iloc[:, 5].tolist()
plot(obese, education, "College Education")
plot(obese, income, "Low Income")
education_corr = correlation(obese, education)
education_corr
income_corr = correlation(obese, income)
income_corr<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0869318986266459, 'fraction_numerical': 0.005238567181084525, 'mean_word_length': 5.645343367826905, 'pattern_counts': {'":': 5, '<': 44, '<?xml version=': 0, '>': 44, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '146689', 'license': 'BSD-3-Clause', 'n_tokens_mistral': 2314, 'n_tokens_neox': 2155, 'n_words': 859, 'path': 'dpyne01/project.ipynb', 'repo_name': 'pezLyfe/applied_ds', 'size': '7063'}