<jupyter_start><jupyter_text>Decision Tree ClassifierHere we will attempt to classify the [*Pima Indians Diabetes Database*](https://www.kaggle.com/uciml/pima-indians-diabetes-database) using the [*Scikit Learn*](https://scikit-learn.org/stable/modules/tree.html) library, and then visualize our decision tree.   **Pros:** 1. *Easy* to interpret and visualize, and require *less* data preprocessing (*eg:* there is no need to normalize columns)  2. Can *easily* capture Nonlinear Patterns  3. *Usefull* in variable selection and in feature engineering (*eg:* predicting missing values)  4. Makes no assumptions about the distribution (due to non parameteric nature of algorithm)  **Cons:**1. Does not support missing values, and can easily overfit noisy data (Pruning required)2. Biased with imbalanced dataset, thus its recommended to [balance the dataset](https://towardsdatascience.com/having-an-imbalanced-dataset-here-is-how-you-can-solve-it-1640568947eb) before creating the tree.Pruning: Method to *prevent overfitting* in Decision Tree by setting the minimum number of samples required at a leaf node or setting the maximum depth of the tree.  References: [datacamp](https://www.datacamp.com/community/tutorials/decision-tree-classification-python)<jupyter_code>%matplotlib inline
import seaborn as sns
from matplotlib import pyplot as plt
import pandas as pd
from sklearn import model_selection, tree, metrics
from graphviz import Source
from ipywidgets import interactive
from IPython.display import display, Image
# Read the dataset (https://www.kaggle.com/uciml/pima-indians-diabetes-database)
data = pd.read_csv('dataset/diabetes.csv')

# Seperate dependant and independant variables
X = data.drop('Outcome', axis='columns')  # Independant Features
y = data['Outcome']  # Dependant Features

print("rows, col: ", data.shape)
display(data.head())<jupyter_output>rows, col:  (768, 9)<jupyter_text>Note: In case values weren't numbers (eg: names), import [*LabelEncoder*](https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html) from [*sklearn.preprocessing*](https://scikit-learn.org/stable/modules/preprocessing.htmlpreprocessing) and use the *fit_transform* function.<jupyter_code># Split Training Data and Test Data for analysing performance (30% test)
X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.3, random_state=1)

# Create Decision Tree Classifer object
clf = tree.DecisionTreeClassifier(criterion="entropy", max_depth=4)  # Pre-pruning by controlling depth
clf.fit(X_train, y_train)  # Train Decision Tree Classifier

y_pred = clf.predict(X_test)  # Predict response for test dataset

print("Accuracy: ", metrics.accuracy_score(y_test, y_pred))  # Print the accuracy on our model

# Plotting a confusion matrix (x-axis = prediction, y-axis = actual ans) to visualise our accuracy
cm = metrics.confusion_matrix(y_test, y_pred)
sns.heatmap(cm, annot=True, annot_kws={"size": 16})
plt.xlabel('Predicted Label', size=18), plt.ylabel('Truth Label', size=18)
# Visualizing our tree, and experimenting with its depth
def plot_tree(depth):
    # Test sample data and calculate and display our accuracy score
    clf = tree.DecisionTreeClassifier(random_state=0, max_depth=depth)
    clf.fit(X_train, y_train)
    y_pred = clf.predict(X_test)
    print("Accuracy:", metrics.accuracy_score(y_test, y_pred))

    # Display our tree in the form of a graph
    graph = Source(tree.export_graphviz(clf, out_file=None, feature_names=X.columns.values,
                                        class_names=['0', '1'], filled=True, rounded=True))
    display(Image(graph.pipe(format='png')))


display(interactive(plot_tree, depth=(1, 10)))  # Interactive IPython slider for depth<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.10647291941875825, 'fraction_numerical': 0.010039630118890357, 'mean_word_length': 5.527586206896552, 'pattern_counts': {'":': 1, '<': 8, '<?xml version=': 0, '>': 8, 'https://': 7, 'lorem ipsum': 0, 'www.': 3, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 1, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '2', 'dir': 'jupyter-structured-clean-dedup', 'id': '552765', 'license': 'MIT', 'n_tokens_mistral': 1204, 'n_tokens_neox': 1138, 'n_words': 347, 'path': 'Decision Tree Classifier/decisionTree.ipynb', 'repo_name': 'relaxxpls/DS303', 'size': '3785'}