<jupyter_start><jupyter_text>Basic Approach for Digit Classifier First let us import the required dependencies like pandas,matplotlib and sklearn.  As pandas library prvides us with wider functionality to wrangle the data, it is very useful.<jupyter_code>import pandas as pd
import matplotlib.pyplot as plt, matplotlib.image as mpimg
from sklearn.model_selection import train_test_split
from sklearn import svm
%matplotlib inline<jupyter_output><empty_output><jupyter_text>Load the Data Next let us use the pandas's read_csv to load the data.  In below repalce the 'path' with your location of the dataset.As below i has considered only the first 5000 training examples, you can take the entire data.<jupyter_code>Location=r'$path\train.csv'
labeled_images = pd.read_csv(Location)
images = labeled_images.iloc[0:5000,1:]
labels = labeled_images.iloc[0:5000,:1]
train_images, test_images,train_labels, test_labels = train_test_split(images, labels, train_size=0.8, random_state=0)<jupyter_output><empty_output><jupyter_text>To view the image, Sinceit is currently one-dimension, we load it into a numpy array and reshape it so that it is two-dimensional (28x28 pixels)Then, we plot the image and label with matplotlib<jupyter_code>i=1
img=train_images.iloc[i].as_matrix()
img=img.reshape((28,28))
plt.imshow(img,cmap='gray')
plt.title(train_labels.iloc[i,0])<jupyter_output><empty_output><jupyter_text>A histogram of the image's pixel values are shown below.<jupyter_code>plt.hist(train_images.iloc[i])<jupyter_output><empty_output><jupyter_text>Training our model First, we use the sklearn.svm module to create a vector classifier.  Next, we pass our training images and labels to the classifier's fit method, which trains our model.  Finally, the test images and labels are passed to the score method to see how well we trained our model. Fit will return a float between 0-1 indicating our accuracy on the test data set<jupyter_code>clf = svm.SVC()
clf.fit(train_images, train_labels.values.ravel())
clf.score(test_images,test_labels)<jupyter_output><empty_output><jupyter_text>You should have gotten around 0.10, or 10% accuracy. This is terrible. 10% accuracy is what get if you randomly guess a number.   There are many ways to improve this, including not using a vector classifier, but here's a simple one to start. Let's just simplify our images by making them true black and white.¶<jupyter_code>test_images[test_images>0]=1
train_images[train_images>0]=1

img=train_images.iloc[i].as_matrix().reshape((28,28))
plt.imshow(img,cmap='binary')
plt.title(train_labels.iloc[i])
plt.hist(train_images.iloc[i])<jupyter_output><empty_output><jupyter_text>Retraining our model We follow the same procedure as before, but now our training and test sets are black and white instead of gray-scale.  Our score still isn't great, but it's a huge improvement.<jupyter_code>clf = svm.SVC()
clf.fit(train_images, train_labels.values.ravel())
clf.score(test_images,test_labels)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.08578104138851803, 'fraction_numerical': 0.015353805073431242, 'mean_word_length': 6.7642487046632125, 'pattern_counts': {'":': 0, '<': 29, '<?xml version=': 0, '>': 31, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '7', 'dir': 'jupyter-structured-clean-dedup', 'id': '132192', 'license': 'MIT', 'n_tokens_mistral': 996, 'n_tokens_neox': 939, 'n_words': 315, 'path': 'Digit classifier using SVM.ipynb', 'repo_name': 'chetanmreddy/Digit-Recognizer', 'size': '2996'}