<jupyter_start><jupyter_text>Heart Disease PredictionIn this machine learning project, I have collected the dataset from Kaggle (https://www.kaggle.com/ronitf/heart-disease-uci) and I will be using Machine Learning algorithms and decide which algorithm is best to make predictions on whether a person is suffering from Heart Disease or not.  Import librariesLet's first import all the necessary libraries. I'll use `numpy` and `pandas` to start with. For visualization, I will use `pyplot` subpackage of `matplotlib`, use `rcParams` to add styling to the plots and `rainbow` for colors. For implementing Machine Learning models and processing of data, I will use the `sklearn` library.<jupyter_code>import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib import rcParams
from matplotlib.cm import rainbow
%matplotlib inline
import warnings
warnings.filterwarnings('ignore')<jupyter_output><empty_output><jupyter_text>For processing the data, I'll import a few libraries. To split the available dataset for testing and training, I'll use the `train_test_split` method. To scale the features, I am using `StandardScaler`.<jupyter_code>from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler<jupyter_output><empty_output><jupyter_text>Next, I'll import all the Machine Learning algorithms I will be using.1. K Neighbors Classifier2. Support Vector Classifier3. Decision Tree Classifier4. Random Forest Classifier<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
from sklearn.svm import SVC
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier<jupyter_output><empty_output><jupyter_text>Import datasetNow that we have all the libraries we will need, I can import the dataset and take a look at it. The dataset is stored in the file `dataset.csv`. I'll use the pandas `read_csv` method to read the dataset.<jupyter_code>dataset = pd.read_csv('c:/users/nikita/desktop/dataset.csv')<jupyter_output><empty_output><jupyter_text>The dataset is now loaded into the variable `dataset`. I'll just take a glimpse of the data using the `desribe()` and `info()` methods before I actually start processing and visualizing it.<jupyter_code>dataset.info()<jupyter_output><class 'pandas.core.frame.DataFrame'>
RangeIndex: 303 entries, 0 to 302
Data columns (total 14 columns):
age         303 non-null int64
sex         303 non-null int64
cp          303 non-null int64
trestbps    303 non-null int64
chol        303 non-null int64
fbs         303 non-null int64
restecg     303 non-null int64
thalach     303 non-null int64
exang       303 non-null int64
oldpeak     303 non-null float64
slope       303 non-null int64
ca          303 non-null int64
thal        303 non-null int64
target      303 non-null int64
dtypes: float64(1), int64(13)
memory usage: 33.2 KB<jupyter_text>Looks like the dataset has a total of 303 rows and there are no missing values. There are a total of `13 features` along with one target value which we wish to find.<jupyter_code>dataset.describe()<jupyter_output><empty_output><jupyter_text>The scale of each feature column is different and quite varied as well. While the maximum for `age` reaches 77, the maximum of `chol` (serum cholestoral) is 564.  Understanding the dataNow, we can use visualizations to better understand our data and then look at any processing we might want to do.<jupyter_code>rcParams['figure.figsize'] = 20, 14
plt.matshow(dataset.corr())
plt.yticks(np.arange(dataset.shape[1]), dataset.columns)
plt.xticks(np.arange(dataset.shape[1]), dataset.columns)
plt.colorbar()<jupyter_output><empty_output><jupyter_text>Taking a look at the correlation matrix above, it's easy to see that a few features have negative correlation with the target value while some have positive.Next, I'll take a look at the histograms for each variable.<jupyter_code>dataset.hist()<jupyter_output><empty_output><jupyter_text>Taking a look at the histograms above, I can see that each feature has a different range of distribution. Thus, using scaling before our predictions should be of great use. Also, the categorical features do stand out. It's always a good practice to work with a dataset where the target classes are of approximately equal size. Thus, let's check for the same.<jupyter_code>rcParams['figure.figsize'] = 8,6
plt.bar(dataset['target'].unique(), dataset['target'].value_counts(), color = ['red', 'green'])
plt.xticks([0, 1])
plt.xlabel('Target Classes')
plt.ylabel('Count')
plt.title('Count of each Target Class')<jupyter_output><empty_output><jupyter_text>The two classes are not exactly 50% each but the ratio is good enough to continue without dropping/increasing our data.  Data ProcessingAfter exploring the dataset, I observed that I need to convert some categorical variables into dummy variables and scale all the values before training the Machine Learning models.First, I'll use the `get_dummies` method to create dummy columns for categorical variables.<jupyter_code>dataset = pd.get_dummies(dataset, columns = ['sex', 'cp', 'fbs', 'restecg', 'exang', 'slope', 'ca', 'thal'])<jupyter_output><empty_output><jupyter_text>Now, I will use the `StandardScaler` from `sklearn` to scale my dataset.<jupyter_code>standardScaler = StandardScaler()
columns_to_scale = ['age', 'trestbps', 'chol', 'thalach', 'oldpeak']
dataset[columns_to_scale] = standardScaler.fit_transform(dataset[columns_to_scale])<jupyter_output><empty_output><jupyter_text>The data is not ready for our Machine Learning application.  Machine LearningI'll now import `train_test_split` to split our dataset into training and testing datasets. Then, I'll import all Machine Learning models I'll be using to train and test the data.<jupyter_code>y = dataset['target']
X = dataset.drop(['target'], axis = 1)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.33, random_state = 0)<jupyter_output><empty_output><jupyter_text>K Neighbors ClassifierThe classification score varies based on different values of neighbors that we choose. Thus, I'll plot a score graph for different values of K (neighbors) and check when do I achieve the best score.<jupyter_code>knn_scores = []
for k in range(1,21):
    knn_classifier = KNeighborsClassifier(n_neighbors = k)
    knn_classifier.fit(X_train, y_train)
    knn_scores.append(knn_classifier.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>I have the scores for different neighbor values in the array `knn_scores`. I'll now plot it and see for which value of K did I get the best scores.<jupyter_code>plt.plot([k for k in range(1, 21)], knn_scores, color = 'red')
for i in range(1,21):
    plt.text(i, knn_scores[i-1], (i, knn_scores[i-1]))
plt.xticks([i for i in range(1, 21)])
plt.xlabel('Number of Neighbors (K)')
plt.ylabel('Scores')
plt.title('K Neighbors Classifier scores for different K values')<jupyter_output><empty_output><jupyter_text>From the plot above, it is clear that the maximum score achieved was `0.87` for the 8 neighbors.<jupyter_code>print("The score for K Neighbors Classifier is {}% with {} nieghbors.".format(knn_scores[7]*100, 8))<jupyter_output>The score for K Neighbors Classifier is 87.0% with 8 nieghbors.<jupyter_text>Support Vector ClassifierThere are several kernels for Support Vector Classifier. I'll test some of them and check which has the best score.<jupyter_code>svc_scores = []
kernels = ['linear', 'poly', 'rbf', 'sigmoid']
for i in range(len(kernels)):
    svc_classifier = SVC(kernel = kernels[i])
    svc_classifier.fit(X_train, y_train)
    svc_scores.append(svc_classifier.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>I'll now plot a bar plot of scores for each kernel and see which performed the best.<jupyter_code>colors = rainbow(np.linspace(0, 1, len(kernels)))
plt.bar(kernels, svc_scores, color = colors)
for i in range(len(kernels)):
    plt.text(i, svc_scores[i], svc_scores[i])
plt.xlabel('Kernels')
plt.ylabel('Scores')
plt.title('Support Vector Classifier scores for different kernels')<jupyter_output><empty_output><jupyter_text>The `linear` kernel performed the best, being slightly better than `rbf` kernel.<jupyter_code>print("The score for Support Vector Classifier is {}% with {} kernel.".format(svc_scores[0]*100, 'linear'))<jupyter_output>The score for Support Vector Classifier is 83.0% with linear kernel.<jupyter_text>Decision Tree ClassifierHere, I'll use the Decision Tree Classifier to model the problem at hand. I'll vary between a set of `max_features` and see which returns the best accuracy.<jupyter_code>dt_scores = []
for i in range(1, len(X.columns) + 1):
    dt_classifier = DecisionTreeClassifier(max_features = i, random_state = 0)
    dt_classifier.fit(X_train, y_train)
    dt_scores.append(dt_classifier.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>I selected the maximum number of features from 1 to 30 for split. Now, let's see the scores for each of those cases.<jupyter_code>plt.plot([i for i in range(1, len(X.columns) + 1)], dt_scores, color = 'green')
for i in range(1, len(X.columns) + 1):
    plt.text(i, dt_scores[i-1], (i, dt_scores[i-1]))
plt.xticks([i for i in range(1, len(X.columns) + 1)])
plt.xlabel('Max features')
plt.ylabel('Scores')
plt.title('Decision Tree Classifier scores for different number of maximum features')<jupyter_output><empty_output><jupyter_text>The model achieved the best accuracy at three values of maximum features, `2`, `4` and `18`.<jupyter_code>print("The score for Decision Tree Classifier is {}% with {} maximum features.".format(dt_scores[17]*100, [2,4,18]))<jupyter_output>The score for Decision Tree Classifier is 79.0% with [2, 4, 18] maximum features.<jupyter_text>Random Forest ClassifierNow, I'll use the ensemble method, Random Forest Classifier, to create the model and vary the number of estimators to see their effect.<jupyter_code>rf_scores = []
estimators = [10, 100, 200, 500, 1000]
for i in estimators:
    rf_classifier = RandomForestClassifier(n_estimators = i, random_state = 0)
    rf_classifier.fit(X_train, y_train)
    rf_scores.append(rf_classifier.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>The model is trained and the scores are recorded. Let's plot a bar plot to compare the scores.<jupyter_code>colors = rainbow(np.linspace(0, 1, len(estimators)))
plt.bar([i for i in range(len(estimators))], rf_scores, color = colors, width = 0.8)
for i in range(len(estimators)):
    plt.text(i, rf_scores[i], rf_scores[i])
plt.xticks(ticks = [i for i in range(len(estimators))], labels = [str(estimator) for estimator in estimators])
plt.xlabel('Number of estimators')
plt.ylabel('Scores')
plt.title('Random Forest Classifier scores for different number of estimators')<jupyter_output><empty_output><jupyter_text>The maximum score is achieved when the total estimators are 100 or 500.<jupyter_code>print("The score for Random Forest Classifier is {}% with {} estimators.".format(rf_scores[1]*100, [100, 500]))<jupyter_output>The score for Random Forest Classifier is 84.0% with [100, 500] estimators.
{'fraction_non_alphanumeric': 0.09641477221673106, 'fraction_numerical': 0.021116003234791986, 'mean_word_length': 5.883116883116883, 'pattern_counts': {'":': 0, '<': 93, '<?xml version=': 0, '>': 93, 'https://': 1, 'lorem ipsum': 0, 'www.': 1, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '24', 'dir': 'jupyter-structured-clean-dedup', 'id': '66223', 'license': 'MIT', 'n_tokens_mistral': 3725, 'n_tokens_neox': 3415, 'n_words': 1217, 'path': 'Heart Disease Prediction.ipynb', 'repo_name': 'nikitaarora497/heartdiseaseprediction', 'size': '11129'}