<jupyter_start><jupyter_text>Lambda School Data Science - Making Data-backed AssertionsThis is, for many, the main point of data science - to create and support reasoned arguments based on evidence. It's not a topic to master in a day, but it is worth some focused time thinking about and structuring your approach to it.  Assignment - what's going on here?Consider the data in `persons.csv` (already prepared for you, in the repo for the week). It has four columns - a unique id, followed by age (in years), weight (in lbs), and exercise time (in minutes/week) of 1200 (hypothetical) people.Try to figure out which variables are possibly related to each other, and which may be confounding relationships.Try and isolate the main relationships and then communicate them using crosstabs and graphs. Share any cool graphs that you make with the rest of the class in Slack!<jupyter_code>!pip install pandas==0.23.4
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import random 
from collections import namedtuple as nt 
person_url='https://raw.githubusercontent.com/LambdaSchool/DS-Unit-1-Sprint-1-Dealing-With-Data/master/module3-databackedassertions/persons.csv'
df1 = pd.read_csv(person_url)
df1.sample(20)
df1.columns[0]
# first column is a redundant and useless index

df1 = df1.drop(['Unnamed: 0'],axis=1)
df1.isna().sum()
df1.describe()<jupyter_output><empty_output><jupyter_text>Exercise time crosstabI could not think of any meaningful crosstabulation for these features.<jupyter_code>?plt.plot()
df1.plot.scatter('age', 'exercise_time')
df1.plot.scatter('weight', 'exercise_time')<jupyter_output><empty_output><jupyter_text>Scatter plotshows that exercise time, if we assume it to mean measure of cardiovascular capacity (instead of, suppose, average daily exercise habit) - people of any age or weight can be 'all over the map' in terms of cardiovascular health.There is no measure of weight loss, so it is quite unreasonable to suppose an inverted relationship between exercise time and weight.It is not possible to assume that exercise time has an influence on age.  Assignment questionsAfter you've worked on some code, answer the following questions in this text block:1.  What are the variable types in the data?> Each observation has age, weight, exercise time ; > * age is independent, continuous but could be treated as ordinal in buckets> * weight is independent, continuous> * excersise_time is apparently dependent, continuous2.  What are the relationships between the variables?> * age and weight are exclusively independent of each other> * exercise_time 'appears' to be dependent3.  Which relationships are "real", and which spurious?> * exercise_time appears to be a function of age and weight but in reality reflects a hidden/missing measure of "cardiovascular health." > * There are certainly some "old" people in good health, and while certainly cardiovascular capacity naturally decreases with age in general, more likely "weight" would be a better indicator of cardiovascular health over all ages -but only if it were possible to know whether the person is obese: without measures of body fat percentage and height, this cannot be inferred. A very "big" or tall person is not necessarily overweight nor in poor cardiovascular health.  Weight and Age bins<jupyter_code>abins = [18, 30, 42, 54, 66, 80]
wbins = [90, 130, 160, 190, 220, 250]
weight_bins = pd.cut(df1['weight'],wbins)
age_bins = pd.cut(df1['age'],abins)
?pd.crosstab #actually not helpful<jupyter_output><empty_output><jupyter_text>https://pandas.pydata.org/pandas-docs/version/0.23.4/generated/pandas.crosstab.htmlInexcusable that agg funcs are not listed or not even a link!Only trivial examples!https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.pivot_table.htmlbetter examples but no guarantee of congruent syntax.<jupyter_code>pd.crosstab(df1['weight'],age_bins) # just gives counts
pd.crosstab(df1['weight'],age_bins,values=df1['exercise_time'], aggfunc=np.mean)<jupyter_output><empty_output><jupyter_text>an improvement but really need that weight bin!<jupyter_code>pd.crosstab(weight_bins,age_bins,values=df1['exercise_time'], aggfunc=np.mean)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07909738717339668, 'fraction_numerical': 0.015914489311163897, 'mean_word_length': 5.770096463022508, 'pattern_counts': {'":': 0, '<': 21, '<?xml version=': 0, '>': 29, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '5', 'dir': 'jupyter-structured-clean-dedup', 'id': '199718', 'license': 'MIT', 'n_tokens_mistral': 1264, 'n_tokens_neox': 1154, 'n_words': 497, 'path': 'module3-databackedassertions/Joseph_Shirk_LS_DS_113_Making_Data_backed_Assertions_Assignment.ipynb', 'repo_name': 'jbshirk/DS-Unit-1-Sprint-1-Dealing-With-Data', 'size': '4210'}