<jupyter_start><jupyter_text>--- **Data Science with Python : Exploratory Data Analysis using Python*** Exploratory Data Analysis(EDA) is the method to get overview of dataset. It is the method of exploring the data and summarizing their main characterstic. * This step is very important especially when we arrive at modeling the data in order to apply Machine learning. * EDA helps determine how best to manipulate data sources to get the answers you need, making it easier for data scientists to discover patterns, spot anomalies, test a hypothesis, or check assumptions.* There is no one method or common methods in order to perform EDA, whereas in this documentation you can understand some common methods and plots that would be used in the EDA process.--- --- **Infromation About Dataset**This is a dataset of breast cancer patient survival who went through surgery. This case study was conducted between 1958-1970. It has 4 attributes as follows.[Download dataset-Kaggle](https://www.kaggle.com/krpiku/haberman.csv)Columns1. age - Age of Patients2. year - Year on which they were operated on3. nodes - number of nodes found4. Status - 1/2           1 - survived less than 5 years           2 - survived more than 5 years--- **Imported important libraries to work on dataset**<jupyter_code>import numpy as np                                                 # Implements milti-dimensional array and matrices
import pandas as pd                                                # For data manipulation and analysis
import matplotlib.pyplot as plt                                    # Plotting library for Python programming language and it's numerical mathematics extension NumPy
import seaborn as sns                                              # Provides a high level interface for drawing attractive and informative statistical graphics
%matplotlib inline<jupyter_output><empty_output><jupyter_text>**DATA EXPLORATION*** Data exploration is the first step towards data analysis. We visualize the data and try to understand it.* We load the dataset and analyze the features and labels present inside the dataset.<jupyter_code>#Loading the data into the dataframe
df=pd.read_csv('/content/haberman breast_cancer.csv')<jupyter_output><empty_output><jupyter_text>**Reading first five values of dataset**<jupyter_code># To display first five rows of the dataset
df.head()<jupyter_output><empty_output><jupyter_text>**Reading last five values of dataset**<jupyter_code># To display last five rows of the dataset
df.tail()
print(df.shape)             #gives the no. of rows and colomns exist in the dataset
print(df.columns)           #provides the name of the columns 
df.nunique()
# checking the datatypes
df.dtypes
print(df['status'].value_counts())  #counts the rows which belongs to class 1 and 2 respectively
# isull() checks is there any null value present in column
#sum() prints the total null value present in any column
df.isnull().sum()
# Dropping duplicate value if there exist any
if df.duplicated().sum()>0:
  df.drop_duplicates(keep="first", inplace=True)
df.shape
# Display concise summary of the dataset
df.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
Int64Index: 288 entries, 0 to 304
Data columns (total 4 columns):
 #   Column  Non-Null Count  Dtype
---  ------  --------------  -----
 0   age     288 non-null    int64
 1   year    288 non-null    int64
 2   nodes   288 non-null    int64
 3   status  288 non-null    int64
dtypes: int64(4)
memory usage: 11.2 KB<jupyter_text>---**OBSERVATIONS CONCLUDED FROM INFO METHOD*** Rows: 288* Columns: 4* Column Name: AGE, YEAR, NODES, STATUS* The dataset is imabalanced* STATUS 1 has 225(LIVED MORE THAN 5 YEARS)* STATUS 2 has 81(LIVED LESS THAN 5 YEARS)---<jupyter_code># Data Description
#describe method gives us the info such as
#count, mean, std, min, max etc
#these details help to give brief idea about the dataset that we are processing 
df.describe()<jupyter_output><empty_output><jupyter_text>---**OBSERVATIONS CONCLUDED FROM DESCRIBE METHOD*** Average age of patients : 52 years* Average year of operation : 1962* Average no. of nodes found in patient:4* RANGE OF COLUMNS :1.   AGE(MIN:30, MAX:78)2.   YEAR(MIN:1958, MAX:1969)3.   NODES(MIN:0, MAX:52)---<jupyter_code># Display the relation between the data means how much one variable (column) is influencing the other variable in the data.
features=df.drop('status', axis=1)
sns.heatmap(features.corr(),annot=True)<jupyter_output><empty_output><jupyter_text>**BIVARIATE ANALYSIS**> Analysis of two variable is known as bivariate analysis. We can plot 2D scatter plots for plotting graph between two variables.<jupyter_code># 2-D Scatter Plot - provides visual image of the relationship between two variables.
sns.set_style("whitegrid");
#scatterplot between age and year
sns.scatterplot(data=df, x="age", y="year",hue='status',palette='muted')
plt.show()
sns.set_style("whitegrid");

#scatterplot between age and nodes
sns.scatterplot(data=df, x="age", y="nodes",hue='status',palette='muted')
plt.show()
sns.set_style("whitegrid");

#scatterplot between year and nodes
sns.scatterplot(data=df, x="year", y="nodes",hue='status',palette='muted')
plt.show()<jupyter_output><empty_output><jupyter_text>---**OBSERVATIONS CONCLUDED FROM BIVARIATE ANALYSIS*** Most of the patients have nodes number less than 5.* Patients with age less than 40 have higher chances of survival and comparatively have less number of nodes.* People with nodes more than 10 and above age 50 have less chances of survival.* There are more no. of patients between age 40-65.---  **PAIRPLOT**> Plot pairwise relationship in a dataset. It is also possible to show a subset of variables or plot different variables on the rows and columns.> Instead of plotting each and every pair seperately, we can use pair plots.<jupyter_code>sns.set_style("whitegrid");
sns.pairplot(df, hue="status", palette="muted");
plt.show()<jupyter_output><empty_output><jupyter_text>---**OBSERVATIONS CONCLUDED FROM PAIR PLOT*** There is sharp decrease in people between number of nodes from 0-4. Most people have nodes near to zero.* People with less number of nodes have more chance of survival and vice-versa.* People above 5 nodes have survived almost half as compared to deaths.---  **BOX-PLOTS**<jupyter_code>sns.boxplot(x='status',y='year', data=df)
plt.show()
sns.boxplot(x='status',y='nodes', data=df)
plt.show()<jupyter_output><empty_output><jupyter_text>---**OBSERVATIONS CONCLUDED FROM BOX-PLOT*** Lower age slighltly indicates high rate of survival.* As year of operation is increasing, there are comparatively little high rate of success.* There are few patients with number of nodes more than 10.* Less the no. of nodes, higher the survival chance.---  **VIOLIN PLOTS**<jupyter_code>sns.violinplot(x="status", y="age", data=df, size=8)

plt.show()
sns.violinplot(x="status", y="year", data=df, size=8)

plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09519009216589862, 'fraction_numerical': 0.017425115207373273, 'mean_word_length': 4.32183908045977, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 45, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 8, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '262761', 'license': 'MIT', 'n_tokens_mistral': 2104, 'n_tokens_neox': 1959, 'n_words': 815, 'path': 'Datascience_With_Python/Data Analysis and Visualization/Exploratory Data Analysis/exploratory_data_analysis.ipynb', 'repo_name': 'vishnupriya129/winter-of-contributing', 'size': '6944'}