<jupyter_start><jupyter_text>Decision Tree ![Decision Tree](Decision-Trees-modified-1.png) In the decision tree we have:- Nodes    - Split for value of a certain attribute.- Edges    - Outcome of a split to next node.- Root    - The node that performs the first split.- Leave    - Terminal node that predict the outcome.  Intuition Behind Split Imaginary Data with 3 features (X,Y and Z) with two possible classes| X | Y | Z | Class ||---|---|---|----|| 1 | 1 | 1 | A || 1 | 1 | 0 | A || 0 | 0 | 1 | B || 1 | 0 | 0 | B |Spliting on Y give us a clear separation between classes.Entropy and Information gain are the mathematical methods of choosing the best split.  Random Forests To improve performance, we can use many trees with a random sample of features choosen as the split.- A new random sample of features is choosen for every simple split.- For classification, m is typically chosen to be the square root of p.<jupyter_code>import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
%matplotlib inline
df = pd.read_csv('kyphosis.csv')
df.head()
df.info()
sns.pairplot(data=df, hue='Kyphosis')
from sklearn.model_selection import train_test_split
X = df.drop('Kyphosis', axis=1)
y = df['Kyphosis']
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)
from sklearn.tree import DecisionTreeClassifier
dtree = DecisionTreeClassifier()
dtree.fit(X_train, y_train)
predictions = dtree.predict(X_test)
from sklearn.metrics import classification_report, confusion_matrix
print(confusion_matrix(y_test, predictions))
print()
print(classification_report(y_test, predictions))
from sklearn.ensemble import RandomForestClassifier
rfc = RandomForestClassifier(n_estimators=100)
rfc.fit(X_train, y_train)
rfc_predictions = rfc.predict(X_test)
print(confusion_matrix(y_test, rfc_predictions))
print()
print(classification_report(y_test, rfc_predictions))
df['Kyphosis'].value_counts()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09790922998470168, 'fraction_numerical': 0.010198878123406425, 'mean_word_length': 5.208860759493671, 'pattern_counts': {'":': 0, '<': 5, '<?xml version=': 0, '>': 5, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '1', 'dir': 'jupyter-structured-clean-dedup', 'id': '396618', 'license': 'MIT', 'n_tokens_mistral': 640, 'n_tokens_neox': 610, 'n_words': 218, 'path': '14.Decision Trees and Random Forests/Decision Trees and Random Forest.ipynb', 'repo_name': 'cris-m/Machine-Learning-Bootcamp', 'size': '1961'}