<jupyter_start><jupyter_text>Data Science Unit 1 Sprint Challenge 1 Loading, cleaning, visualizing, and analyzing dataIn this sprint challenge you will look at a dataset of the survival of patients who underwent surgery for breast cancer.http://archive.ics.uci.edu/ml/datasets/Haberman%27s+SurvivalData Set Information:The dataset contains cases from a study that was conducted between 1958 and 1970 at the University of Chicago's Billings Hospital on the survival of patients who had undergone surgery for breast cancer.Attribute Information:1. Age of patient at time of operation (numerical)2. Patient's year of operation (year - 1900, numerical)3. Number of positive axillary nodes detected (numerical)4. Survival status (class attribute)-- 1 = the patient survived 5 years or longer-- 2 = the patient died within 5 yearSprint challenges are evaluated based on satisfactory completion of each part. It is suggested you work through it in order, getting each aspect reasonably working, before trying to deeply explore, iterate, or refine any given step. Once you get to the end, if you want to go back and improve things, go for it!  Part 1 - Load and validate the data- Load the data as a `pandas` data frame. (DONE)- Validate that it has the appropriate number of observations (you can check the raw file, and also read the dataset description from UCI).(DONE)- Validate that you have no missing values.(DONE)- Add informative names to the features.(DONE)- The survival variable is encoded as 1 for surviving >5 years and 2 for not - change this to be 0 for not surviving and 1 for surviving >5 years (0/1 is a more traditional encoding of binary variables)(DONE)At the end, print the first five rows of the dataset to demonstrate the above.<jupyter_code># Loading the data as a pandas data frame
import pandas as pd
import numpy as np<jupyter_output><empty_output><jupyter_text>**Attributes**Age = AgeYOO = Year Of OperationPAND = Positive Axillary Nodes DetectedSS = Survival Status **Read in the dataset, assign names to features and verify it matches the DS**<jupyter_code>patient_data = pd.read_csv('http://archive.ics.uci.edu/ml/machine-learning-databases/haberman/haberman.data', 
                           header=None, 
                          names=['Age', 'YOO', 'PAND', 'SS'])

patient_data.head()

# Validate appropriate number of observations
#patient_data.count()<jupyter_output><empty_output><jupyter_text>**Verify you have no missing values, this can be done in a couple of ways, which I have listed in hastags below. i chose to use my favorite method that just shows a sum of all missing values**<jupyter_code># Verify you have no missing values. 
#patient_data.isnull().sum()
#patient_data.isna().sum()

patient_data.isnull().sum().sum()<jupyter_output><empty_output><jupyter_text>**Change the Variables and Print the First 5 Rows**<jupyter_code>patient_data['SS']= np.where(patient_data['SS']==1,1,0)
# Confirm
patient_data.head()<jupyter_output><empty_output><jupyter_text>Part 2 - Examine the distribution and relationships of the featuresExplore the data - create at least *2* tables (can be summary statistics or crosstabulations) and *2* plots illustrating the nature of the data.This is open-ended, so to remind - first *complete* this task as a baseline, then go on to the remaining sections, and *then* as time allows revisit and explore further.Hint - you may need to bin some variables depending on your chosen tables/plots. **Crosstabs & Summary Stats**<jupyter_code># Making Bins to see if there is any connections
yoo_bins = pd.cut(patient_data['YOO'], 10)  # 10 Bins
age_bins = pd.cut(patient_data['Age'], 10)  # 10 Bins
pand_bins = pd.cut(patient_data['PAND'], 10) # 10 Bins


ct = pd.crosstab(yoo_bins,[age_bins,patient_data['SS']],
                  rownames=['Year of Op'],colnames=['Age','Survived'])
ct.head()<jupyter_output><empty_output><jupyter_text>***I did not see any correlation between Year of operation and age/survival. To properly gauge this, I would need more info.***<jupyter_code>ct = pd.crosstab(pand_bins,[age_bins,patient_data['SS']],
                  rownames=['Nodes'],colnames=['Age','Survived'])
ct.head()<jupyter_output><empty_output><jupyter_text>***I enjoyed this chart. It shows the relationship between age, survival and nodes. I felt like it showed a lot of insight, although a bit messy.***<jupyter_code>ct.describe()<jupyter_output><empty_output><jupyter_text>**Plots**<jupyter_code># Bar plot
patient_data['SS'].value_counts().plot(kind='bar')
plt.xlabel('Survival')
plt.ylabel('Patients')
plt.title('Survival Rates vs Patients')
plt.show()<jupyter_output><empty_output><jupyter_text>***This plot does a great job representing the data. This one in particular shows the relationship between patients and survival. ***<jupyter_code># Bar Plot #2
patient_data['Age'].value_counts().plot(kind='bar')
plt.xlabel('Age')
plt.ylabel('Survival')
plt.title('Survival Rates vs Ages')
plt.show()<jupyter_output><empty_output><jupyter_text>***This is a bit cluttered on the X axis, could use some work with more time. ***<jupyter_code>#Bar Plot Pink
pd.pivot_table(patient_data, values='SS', index=age_bins).plot.bar(color= 'pink'); 
plt.xlabel('Age', color='pink', size=20)
plt.ylabel('Survival', color='pink', size=20)
plt.title('Plot #1: Age vs Survival', color='hotpink', size=25);

pd.pivot_table(patient_data, values='Age', index=pand_bins).plot.bar(color= 'pink'); 
plt.xlabel('Age', color='pink', size=20)
plt.ylabel('Survival', color='pink', size=20)
plt.title('Plot #2: Age vs Nodes Detected', color='hotpink', size=25);

pd.pivot_table(patient_data, values='PAND', index='SS').plot.bar(color= 'pink'); 
plt.xlabel('Survival Class', color='pink', size=20)
plt.ylabel('Nodes', color='pink', size=20)
plt.title('Plot #3: Survival vs Nodes Detected', color='hotpink', size=25);<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.11345466916142201, 'fraction_numerical': 0.012587174689573056, 'mean_word_length': 5.629086809470124, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 43, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '294136', 'license': 'MIT', 'n_tokens_mistral': 1894, 'n_tokens_neox': 1737, 'n_words': 641, 'path': 'DS_Unit_1_Sprint_Challenge_1.ipynb', 'repo_name': 'tbradshaw91/DS-Unit-1-Sprint-1-Dealing-With-Data', 'size': '5879'}