<jupyter_start><jupyter_text>MNIST[THE MNIST DATABASE of handwritten digits](http://yann.lecun.com/exdb/mnist/)Train classifiers using sklearn to recognize digit images ---  Table of Content- [Setup](setup)- [Load Data](load-data)- [Training](training)    - [Naive Bayes](naive-bayes)    - [SVM](svm)    - [KNN](knn) ---  Setup<jupyter_code>%matplotlib inline
import math
import random

import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import accuracy_score<jupyter_output><empty_output><jupyter_text>---  Load Data<jupyter_code>from sklearn.datasets import load_digits
dataset = load_digits()
dataset
images = dataset.images
labels = dataset.target
print('images.shape: ', images.shape)
print('labels.shape: ', labels.shape)
print('First image:\n', images[0])
print('First label:', labels[0])
images_and_labels = list(zip(images, labels))
display_num = 10
for index, (image, label) in enumerate(images_and_labels[:display_num]):
    plt.subplot(2, math.floor(display_num/2), index + 1)
    plt.axis('off')
    plt.imshow(image, interpolation='nearest')
    plt.title('Training: %i' % label)<jupyter_output><empty_output><jupyter_text>Split training data and testing data<jupyter_code>from sklearn.model_selection import train_test_split
flat_images = images.reshape((1797, 64))
X_train, X_test, y_train, y_test = train_test_split(flat_images, labels, random_state=42)
print('X_train[0]: ', X_train[0])
print('y_train[0]: ', y_train[0])<jupyter_output>X_train[0]:  [ 0.  0.  2. 15. 15. 16. 11.  0.  0.  0.  8. 16. 11.  3.  0.  0.  0.  0.
 13.  9.  0.  0.  0.  0.  0.  5. 16.  3.  9. 11.  3.  0.  0. 10. 15. 15.
 16. 16. 11.  0.  0.  6. 16. 10.  7. 16.  5.  0.  0.  0.  3.  4. 15.  8.
  0.  0.  0.  0.  4. 15.  7.  0.  0.  0.]
y_train[0]:  5<jupyter_text>---  Training  Naive Bayes<jupyter_code>from sklearn.naive_bayes import GaussianNB
%%time

clf = GaussianNB()
clf.fit(X_train, y_train)
sampled_image_label = random.sample(list(zip(flat_images, labels)), 1)[0]

print('Image:\n', sampled_image_label[0].reshape(8, 8))
print('Label: ', sampled_image_label[1])
predict_value = clf.predict(sampled_image_label[0].reshape(1, -1))

print('Predict Value: ', predict_value[0])
print('Expected Value: ', sampled_image_label[1])
pred = clf.predict(X_test)
accuracy_score(pred, y_test)<jupyter_output><empty_output><jupyter_text>Reference[1.9. Naive Bayes](http://scikit-learn.org/stable/modules/naive_bayes.html)  SVM<jupyter_code>from sklearn import svm
%%time

clf = svm.SVC(kernel='linear')
clf.fit(X_train, y_train)
pred = clf.predict(X_test)
accuracy_score(y_test, pred)<jupyter_output><empty_output><jupyter_text>Deploy an RBF Kernel<jupyter_code>%%time

clf = svm.SVC(kernel='rbf', C=1000, gamma='scale')
clf.fit(X_train, y_train)
pred = clf.predict(X_test)
accuracy_score(y_test, pred)<jupyter_output><empty_output><jupyter_text>Find the best parameter<jupyter_code>from sklearn.model_selection import GridSearchCV
svc = svm.SVC()
parameters = {
    'kernel': ('linear', 'poly', 'rbf', 'sigmoid'),
    'C': (1, 10, 100),
    'gamma': ('scale',)
}
%%time

clf = GridSearchCV(svc, parameters, cv=5)
clf.fit(X_train, y_train)
clf.best_estimator_
pred = clf.predict(X_test)
accuracy_score(y_test, pred)<jupyter_output><empty_output><jupyter_text>Reference- [1.4. Support Vector Machine](http://scikit-learn.org/stable/modules/svm.html)- [sklearn.model_selection.GridSearchCV](http://scikit-learn.org/stable/modules/generated/sklearn.model_selection.GridSearchCV.html)  KNN<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
%%time

clf = KNeighborsClassifier(n_neighbors=5)
clf.fit(X_train, y_train)
pred = clf.predict(X_test)
accuracy_score(y_test, pred)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1538047862328583, 'fraction_numerical': 0.03576230169400377, 'mean_word_length': 5.315789473684211, 'pattern_counts': {'":': 0, '<': 32, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '364835', 'license': 'MIT', 'n_tokens_mistral': 1566, 'n_tokens_neox': 1478, 'n_words': 284, 'path': '2 - MNIST.ipynb', 'repo_name': 'Lee-W/sklearn_intro', 'size': '3719'}