<jupyter_start><jupyter_text>2A.ML101.3: Supervised Learning: Classification of Handwritten DigitsIn this section we'll apply scikit-learn to the classification of handwrittendigits.  This will go a bit beyond the iris classification we saw before: we'lldiscuss some of the metrics which can be used in evaluating the effectivenessof a classification model. *Source:* [Course on machine learning with scikit-learn](https://github.com/GaelVaroquaux/sklearn_ensae_course) by <NAME><jupyter_code>from sklearn.datasets import load_digits
digits = load_digits()<jupyter_output><empty_output><jupyter_text>We'll re-use some of our code from before to visualize the data and remind us whatwe're looking at:<jupyter_code>%matplotlib inline
from matplotlib import pyplot as plt
fig = plt.figure(figsize=(6, 6))  # figure size in inches
fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)

# plot the digits: each image is 8x8 pixels
for i in range(64):
    ax = fig.add_subplot(8, 8, i + 1, xticks=[], yticks=[])
    ax.imshow(digits.images[i], cmap=plt.cm.binary, interpolation='nearest')
    
    # label the image with the target value
    ax.text(0, 7, str(digits.target[i]))<jupyter_output><empty_output><jupyter_text>Visualizing the Data A good first-step for many problems is to visualize the data using a*Dimensionality Reduction* technique.  We'll start with themost straightforward one, Principal Component Analysis (PCA).PCA seeks orthogonal linear combinations of the features which show the greatestvariance, and as such, can help give you a good idea of the structure of thedata set.  Here we'll use `RandomizedPCA`, because it's faster for large `N`.<jupyter_code>from sklearn.decomposition import RandomizedPCA
pca = RandomizedPCA(n_components=2)
proj = pca.fit_transform(digits.data)
plt.scatter(proj[:, 0], proj[:, 1], c=digits.target)
plt.colorbar()<jupyter_output><empty_output><jupyter_text>**Question: Given these projections of the data, which numbers do you thinka classifier might have trouble distinguishing?**  Gaussian Naive Bayes Classification For most classification problems, it's nice to have a simple, fast, go-tomethod to provide a quick baseline classification.  If the simple and fastmethod is sufficient, then we don't have to waste CPU cycles on more complexmodels.  If not, we can use the results of the simple method to give usclues about our data.One good method to keep in mind is Gaussian Naive Bayes.  It fits a Gaussian distribution to each training label independantly on each feature, and uses this to quickly give a rough classification.  It is generally not sufficiently accurate for real-world data, but can perform surprisingly well, for instance on text data.<jupyter_code>from sklearn.naive_bayes import GaussianNB
from sklearn.model_selection import train_test_split
# split the data into training and validation sets
X_train, X_test, y_train, y_test = train_test_split(digits.data, digits.target)

# train the model
clf = GaussianNB()
clf.fit(X_train, y_train)

# use the model to predict the labels of the test data
predicted = clf.predict(X_test)
expected = y_test<jupyter_output><empty_output><jupyter_text>**Question**: why did we split the data into training and validation sets? Let's plot the digits again with the predicted labels to get an idea ofhow well the classification is working:<jupyter_code>fig = plt.figure(figsize=(6, 6))  # figure size in inches
fig.subplots_adjust(left=0, right=1, bottom=0, top=1, hspace=0.05, wspace=0.05)

# plot the digits: each image is 8x8 pixels
for i in range(64):
    ax = fig.add_subplot(8, 8, i + 1, xticks=[], yticks=[])
    ax.imshow(X_test.reshape(-1, 8, 8)[i], cmap=plt.cm.binary,
              interpolation='nearest')
    
    # label the image with the target value
    if predicted[i] == expected[i]:
        ax.text(0, 7, str(predicted[i]), color='green')
    else:
        ax.text(0, 7, str(predicted[i]), color='red')<jupyter_output><empty_output><jupyter_text>Quantitative Measurement of Performance We'd like to measure the performance of our estimator without having to resortto plotting examples.  A simple method might be to simply compare the number ofmatches:<jupyter_code>matches = (predicted == expected)
print(matches.sum())
print(len(matches))
matches.sum() / float(len(matches))<jupyter_output><empty_output><jupyter_text>We see that nearly 1500 of the 1800 predictions match the input.  But there are othermore sophisticated metrics that can be used to judge the performance of a classifier:several are available in the ``sklearn.metrics`` submodule.One of the most useful metrics is the ``classification_report``, which combines severalmeasures and prints a table with the results:<jupyter_code>from sklearn import metrics
print(metrics.classification_report(expected, predicted))<jupyter_output>precision    recall  f1-score   support

          0       1.00      0.95      0.98        42
          1       0.60      0.77      0.67        43
          2       1.00      0.60      0.75        48
          3       0.95      0.70      0.80        50
          4       0.95      0.89      0.92        45
          5       0.80      0.89      0.84        45
          6       0.93      1.00      0.96        39
          7       0.69      1.00      0.82        41
          8       0.58      0.61      0.60        51
          9       0.84      0.78      0.81        46

avg / total       0.83      0.81      0.81       450<jupyter_text>Another enlightening metric for this sort of multi-label classificationis a *confusion matrix*: it helps us visualize which labels arebeing interchanged in the classification errors:<jupyter_code>print(metrics.confusion_matrix(expected, predicted))<jupyter_output>[[40  0  0  0  2  0  0  0  0  0]
 [ 0 33  0  0  0  0  2  1  4  3]
 [ 0  6 29  0  0  1  0  0 11  1]
 [ 0  1  0 35  0  4  0  2  6  2]
 [ 0  0  0  0 40  1  0  4  0  0]
 [ 0  1  0  1  0 40  0  2  1  0]
 [ 0  0  0  0  0  0 39  0  0  0]
 [ 0  0  0  0  0  0  0 41  0  0]
 [ 0 11  0  0  0  2  0  6 31  1]
 [ 0  3  0  1  0  2  1  3  0 36]]
{'fraction_non_alphanumeric': 0.07996051332675222, 'fraction_numerical': 0.05067456400131622, 'mean_word_length': 3.302193913658882, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 1, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 10, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '327702', 'license': 'MIT', 'n_tokens_mistral': 2057, 'n_tokens_neox': 1895, 'n_words': 731, 'path': '_doc/notebooks/sklearn_ensae_course/03_supervised_classification.ipynb', 'repo_name': 'mohamedelkansouli/Ensae_py', 'size': '6086'}