<jupyter_start><jupyter_text>Ch6 Decision TreesUsefulf or both classification and regression, and can also do multioutput.<jupyter_code># Where to save the figures
PROJECT_ROOT_DIR = "."
CHAPTER_ID = "decision_trees"
import os

def image_path(fig_id):
    return os.path.join(PROJECT_ROOT_DIR, fig_id)


#Train some trees
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier

iris = load_iris()
X = iris.data[:, 2:] #petal length and width
y = iris.target

tree_clf = DecisionTreeClassifier(max_depth=2)
tree_clf.fit(X,y)
#Visualize tree using graphviz
from sklearn.tree import export_graphviz
export_graphviz(tree_clf, 
                out_file=image_path("iris_tree.dot"), 
                feature_names=iris.feature_names[2:],
                class_names=iris.target_names,
                rounded=True,
                filled=True
               )

# On linux the "XDot" program works great for viewing .dot files<jupyter_output><empty_output><jupyter_text>Tip: A nice thing about decision trees is they are not super sensitive to data preparation (aka don't have a huge need for feature scaling or centering).CART algorithm for trees produce only binary, yes no questions at the nodes. ID3 and others can go beyond binary.  Estimating Class Probabilities<jupyter_code>tree_clf.predict_proba([[5,1.5]])
tree_clf.predict([[5,1.5]])<jupyter_output><empty_output><jupyter_text>CART Training Algorithmaka "growing" trees.1. Split the training data into two groups using feature *k* and a threshold *t_k* (i.e. petal length < 2.45cm).2. How to choose *k* and *t_k*? It searches for values that create purest subsets weighted by size, and uses a cost function.3. Repeat with remaining subsets and keep splitting up to max_depth. CART is Greedy.Computing optimal tree is NP-Complete.  Decision Tree Regression<jupyter_code>from sklearn.tree import DecisionTreeRegressor
tree_reg = DecisionTreeRegressor(max_depth=2)
tree_reg.fit(X,y)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.07973921765295888, 'fraction_numerical': 0.009027081243731194, 'mean_word_length': 4.651558073654391, 'pattern_counts': {'":': 0, '<': 14, '<?xml version=': 0, '>': 13, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '3', 'dir': 'jupyter-structured-clean-dedup', 'id': '50830', 'license': 'MIT', 'n_tokens_mistral': 659, 'n_tokens_neox': 621, 'n_words': 197, 'path': 'handsOn/ch6/DecisionTrees.ipynb', 'repo_name': 'codyhanson/ml-toys', 'size': '1994'}