<jupyter_start><jupyter_text>*This notebook contains an excerpt from the book [Machine Learning for OpenCV](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv) by <NAME>.The code is released under the [MIT license](https://opensource.org/licenses/MIT),and is available on [GitHub](https://github.com/mbeyeler/opencv-machine-learning).**Note that this excerpt contains only the raw code - the book is rich with additional explanations and illustrations.If you find this content useful, please consider supporting the work by[buying the book](https://www.packtpub.com/big-data-and-business-intelligence/machine-learning-opencv)!*   Using Decision Trees to Diagnose Breast CancerNow that we have built our first decision trees, it's time to turn our attention to a real dataset: The Breast Cancer Wisconsin dataset .In order to make the take feasible, the researchers performed feature extraction on the images, like we did in Chapter 4, Representing Data and Engineering Features. They went through a total of 569 images, and extracted 30 different features that describe the characteristics of the cell nuclei present in the images, including:- cell nucleus texture (represented by the standard deviation of the gray-scale values)- cell nucleus size (calculated as the mean of distances from center to points on the perimeter)- tissue smoothness (local variation in radius lengths)- tissue compactnessThe goal of the research was then to classify tissue samples into benign and malignant (a binary classification task).  Loading the datasetThe full dataset is part of Scikit-Learn's example datasets:<jupyter_code>from sklearn import datasets
data = datasets.load_breast_cancer()<jupyter_output><empty_output><jupyter_text>As in previous examples, all data is contained in a 2-D feature matrix data.data, where the rows represent data samples, and the columns are the feature values:<jupyter_code>data.data.shape<jupyter_output><empty_output><jupyter_text>With a look at the provided feature names, we recognize some that we mentioned above:<jupyter_code>data.feature_names<jupyter_output><empty_output><jupyter_text>Since this is a binary classification task, we expect to find exactly two target names:<jupyter_code>data.target_names<jupyter_output><empty_output><jupyter_text>Let's split the dataset into training and test sets using a healthy 80-20 split:<jupyter_code>import sklearn.model_selection as ms
X_train, X_test, y_train, y_test = ms.train_test_split(data.data, data.target, test_size=0.2, random_state=42)
X_train.shape, X_test.shape<jupyter_output><empty_output><jupyter_text>Building the decision tree<jupyter_code>from sklearn import tree
dtc = tree.DecisionTreeClassifier(random_state=42)
dtc.fit(X_train, y_train)<jupyter_output><empty_output><jupyter_text>Since we did not specify any pre-pruning parameters, we would expect this decision tree to grow quite large and result in a perfect score on the training set:<jupyter_code>dtc.score(X_train, y_train)<jupyter_output><empty_output><jupyter_text>However, to our surprise, the test error is not too shabby, either:<jupyter_code>dtc.score(X_test, y_test)
with open("tree.dot", 'w') as f:
    f = tree.export_graphviz(dtc, out_file=f,
                             feature_names=data.feature_names,
                             class_names=data.target_names)<jupyter_output><empty_output><jupyter_text>Now we want to do some model exploration. For example, we mentioned above that the depth of a tree influences its performance. If we wanted to study this dependency more systematically, we could repeat building the tree for different values of `max_depth`:<jupyter_code>import numpy as np
max_depths = np.array([1, 2, 3, 5, 7, 9, 11])<jupyter_output><empty_output><jupyter_text>For each of these values, we want to run the full model cascade from start to finish. We also want to record the train and test scores. We do this in a for loop:<jupyter_code>train_score = []
test_score = []
for d in max_depths:
    dtc = tree.DecisionTreeClassifier(max_depth=d, random_state=42)
    dtc.fit(X_train, y_train)
    train_score.append(dtc.score(X_train, y_train))
    test_score.append(dtc.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>We can plot the scores as a function of the tree depth using Matplotlib:<jupyter_code>import matplotlib.pyplot as plt
%matplotlib inline
plt.style.use('ggplot')
plt.figure(figsize=(10, 6))
plt.plot(max_depths, train_score, 'o-', linewidth=3, label='train')
plt.plot(max_depths, test_score, 's-', linewidth=3, label='test')
plt.xlabel('max_depth')
plt.ylabel('score')
plt.ylim(0.85, 1.1)
plt.legend()<jupyter_output><empty_output><jupyter_text>Let's do one more. What about the minimum numbers of samples required to make a node a leaf node?We repeat the procedure from above:<jupyter_code>train_score = []
test_score = []
min_samples = np.array([2, 4, 8, 16, 32])
for s in min_samples:
    dtc = tree.DecisionTreeClassifier(min_samples_leaf=s, random_state=42)
    dtc.fit(X_train, y_train)
    train_score.append(dtc.score(X_train, y_train))
    test_score.append(dtc.score(X_test, y_test))<jupyter_output><empty_output><jupyter_text>This leads to a plot that looks quite different from the one before:<jupyter_code>plt.figure(figsize=(10, 6))
plt.plot(min_samples, train_score, 'o-', linewidth=3, label='train')
plt.plot(min_samples, test_score, 's-', linewidth=3, label='test')
plt.xlabel('min_samples_leaf')
plt.ylabel('score')
plt.ylim(0.9, 1)
plt.legend()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.09126984126984126, 'fraction_numerical': 0.00974025974025974, 'mean_word_length': 6.072704081632653, 'pattern_counts': {'":': 0, '<': 54, '<?xml version=': 0, '>': 54, 'https://': 4, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '496379', 'license': 'MIT', 'n_tokens_mistral': 1795, 'n_tokens_neox': 1701, 'n_words': 554, 'path': 'notebooks/05.02-Using-Decision-Trees-to-Diagnose-Breast-Cancer.ipynb', 'repo_name': 'qabacus/opencv-machine-learning', 'size': '5553'}