<jupyter_start><jupyter_text>Describe() method generates descriptive statistics that summarize the central tendency, dispersion and shape of a dataset’s distribution, excluding NaN values.This method deals with only numeric columns. To look for categorical variables as well use include="all".count - tells count of the columns or tells number of non empty rowsmean - mean of the columnstd - standard deviation of the columnmin - minimum value of the column25% - 25th percentile/quarter that is median of first and second quarter of numbers50% - median of numbers75% - 75th percentile/quarter that is median of third and fourth quartermax - maximum value of the column<jupyter_code>heart.describe()
continuous_columns = ['creatinine_phosphokinase','ejection_fraction',
                       'platelets','serum_creatinine','serum_sodium','time']
# Check for null values. Dataset does not has any null values.
heart.isnull().sum()
p=heart[continuous_columns].hist(figsize = (20,20))<jupyter_output><empty_output><jupyter_text>We can see skewness for few columns. More about skewness can be read from:https://www.statisticshowto.com/probability-and-statistics/skewed-distribution/<jupyter_code># Finding outliers
# Using sns boxplot
plt.figure(figsize=(15,5))
sns.boxplot(x='variable', y='value', data=pd.melt(heart[continuous_columns]))
fig, ax = plt.subplots(nrows = 2, ncols = 3)
fig.tight_layout()
plt.figure(figsize=(20,10))
i=0
for row in range(0,2):
    for col in range(0,3):
        ax[row,col].boxplot(heart[continuous_columns[i]])
        ax[row,col].set_title(continuous_columns[i])
        i = i+1
plt.show()
heart['log_creatinine_phosphokinase'] = np.log(heart['creatinine_phosphokinase'])
sns.boxplot(data=heart['log_creatinine_phosphokinase'])
heart['log_creatinine_phosphokinase'].hist()
# Check for outliers 
# Outliers are the values which are 3 standard deviations away from mean

z = np.abs(stats.zscore(heart[continuous_columns]))
z.shape
heart.shape
heart.DEATH_EVENT.value_counts(normalize=True)
x_ticks=np.arange(0,9000,100)
plt.figure(figsize=(20, 10))
plt.xticks(x_ticks, rotation=90)
plt.scatter('creatinine_phosphokinase','age', data=heart)
#'serum_creatinine','platelets'
plt.hist( 'creatinine_phosphokinase', data=heart)
heart[heart.creatinine_phosphokinase==7861]
heart.DEATH_EVENT.value_counts()
heart[heart.creatinine_phosphokinase>=600].shape
# Normal range of CPK is in between 20-200. This is a enzyme that catalyzes the reaction of 
# creatine and adenosine triphosphate (ATP) to phosphocreatine and adenosine diphosphate (ADP)
# Mainly available in cardiac, skeletal muscles. If these muscles are damaged then this enzyme leaks in 
# blood stream. Thus. CPK is indication of muscles damage.
heart[(heart.creatinine_phosphokinase>=200)&(heart.DEATH_EVENT==1)].shape
plt.scatter( 'age', 'creatinine_phosphokinase', data=heart, color='darkblue',s=1)
plt.scatter('age', 'serum_creatinine', data=heart, color='red', s=4)<jupyter_output><empty_output><jupyter_text>Let's analyse serum cratinine Source - https://pubmed.ncbi.nlm.nih.gov/9056611/Elevated serum creatinine has been associated with increased mortality in hypertensive persons, the elderly, and patients with myocardial infarction or stroke in whom cardiovascular disease is the major cause of death. We have examined the relationship between serum creatinine concentration and the risk of major ischemic heart disease and stroke events and all-cause mortality in a general population of middle-aged men.<jupyter_code>plt.hist('serum_creatinine', data=heart)
x_ticks=np.arange(0,10,0.5)
plt.xticks(x_ticks)
plt.scatter('serum_creatinine','DEATH_EVENT',data=heart)
heart[(heart.DEATH_EVENT==1) & (heart.serum_creatinine >=1.3) & (heart.high_blood_pressure==1)].shape<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10770855332629356, 'fraction_numerical': 0.020591341077085535, 'mean_word_length': 6.562874251497006, 'pattern_counts': {'":': 0, '<': 13, '<?xml version=': 0, '>': 16, 'https://': 2, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '300942', 'license': 'Apache-2.0', 'n_tokens_mistral': 1315, 'n_tokens_neox': 1142, 'n_words': 327, 'path': 'Data-Analysis-And-Visualization/Data Analysis and Visualization.ipynb', 'repo_name': 'writetoneeraj/ml-with-python', 'size': '3788'}