<jupyter_start><jupyter_text>CIFAR10 classification using HOG features  Load CIFAR10 dataset<jupyter_code>import myutils

raw_data_training, raw_data_testing = myutils.load_CIFAR_dataset(shuffle=False)
# raw_data_training = raw_data_training[:5000]

class_names = myutils.load_CIFAR_classnames()

n_training = len( raw_data_training )
n_testing = len( raw_data_testing )
print('Loaded CIFAR10 database with {} training and {} testing samples'.format(n_training, n_testing))<jupyter_output>Loaded CIFAR10 database with 50000 training and 10000 testing samples<jupyter_text>Preprocessing<jupyter_code># Converting to greyscale
def rgb2gray(image):
    import cv2
    return cv2.cvtColor(image,cv2.COLOR_BGR2GRAY)

Xdata_training = [ rgb2gray(raw_data_training[i][0]) for i in range(n_training)]
Xdata_testing  = [ rgb2gray(raw_data_testing[i][0]) for i in range(n_testing)]<jupyter_output><empty_output><jupyter_text>Lets look how images looks like<jupyter_code>import random
import matplotlib.pyplot as plt
%matplotlib inline

# lets choose some random sample of 10 training images
examples_id = random.sample(range(n_training), 10)
fig, axarr = plt.subplots(2,len(examples_id), figsize=(15,3))
for i in range(len(examples_id)):
    id = examples_id[i]
    axarr[0,i].imshow(raw_data_training[id][0][:,:])
    axarr[0,i].axis('off')
    axarr[1,i].imshow(Xdata_training[id],cmap='gray')
    axarr[1,i].axis('off')
print('Few examples after preprocessing')
plt.show()<jupyter_output>Few examples after preprocessing<jupyter_text>Feature extractionWe use HOG descriptor from [scikit-image](http://scikit-image.org/docs/dev/api/skimage.feature.htmlskimage.feature.hog) library.<jupyter_code># Configuring HOG descriptor
# see http://scikit-image.org/docs/dev/api/skimage.feature.html#skimage.feature.hog

# Configuration of HOG descriptor
normalize = True          #  True ==> yields a little bit better score
                          #  
block_norm = 'L2-Hys'     # or 'L1'
orientations = 9          # 
pixels_per_cell = [8, 8]  #  see section 'Additional remarks' for some explanation
cells_per_block = [2, 2]  # 

def extractFeature(img, vis=False):
    from skimage.feature import hog
    return hog(img, orientations, pixels_per_cell, cells_per_block, block_norm, visualise=vis, transform_sqrt=normalize)<jupyter_output><empty_output><jupyter_text>Visualization of HOG histograms<jupyter_code># extracting one sample data
nfeatures = extractFeature(Xdata_training[0], vis=False).size
print('Number of features = {}'.format(nfeatures))

fig, axarr = plt.subplots(3,len(examples_id), figsize=(16,5))
for i in range(len(examples_id)):
    id = examples_id[i]
    axarr[0,i].imshow(raw_data_training[id][0][:,:])
    axarr[0,i].axis('off')
    axarr[1,i].imshow(Xdata_training[id],cmap='gray')
    axarr[1,i].axis('off')
    _, hog_vis = extractFeature(Xdata_training[id], vis=True)
    axarr[2,i].imshow(hog_vis,cmap='gray')
    axarr[2,i].axis('off')
plt.show()
# feature extraction
import numpy as np
X_training = np.array( [ extractFeature(Xdata_training[i], vis=False) for i in range(n_training) ] )
y_training = np.array( [ raw_data_training[i][1] for i in range(n_training) ] )

X_testing = np.array( [ extractFeature(Xdata_testing[i], vis=False) for i in range(n_testing) ] )
y_testing = np.array( [ raw_data_testing[i][1] for i in range(n_testing) ] )<jupyter_output><empty_output><jupyter_text>Short look at the training data<jupyter_code>print( 'X_training shape is {}'.format( X_training.shape ) )
print( 'y_training shape is {}'.format( y_training.shape ) )
print( 'X_testing shape is {}'.format( X_testing.shape ) )
print( 'y_testing shape is {}'.format( y_testing.shape ) )
import pandas as pd
print( 'X_training data description')
pd.DataFrame( X_training ).describe()
print( 'y_training data description')
pd.DataFrame( y_training ).describe()<jupyter_output>y_training data description<jupyter_text>PCA 2D decomposition<jupyter_code>from sklearn import decomposition
pca = decomposition.PCA(n_components=2)
pca.fit(X_training)
X = pca.transform(X_training)
print(pca.explained_variance_ratio_)

plt.figure( figsize=(15,15) )
plt.scatter( X[:, 0], X[:, 1], c=y_training, cmap='tab10' )
# plt.colorbar()
plt.show()
# TODO: remove outliers<jupyter_output><empty_output><jupyter_text>Classification We simply use *LinearSVC* from [scikit-learn](http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.htmlsklearn.svm.SVC) library.<jupyter_code>from sklearn.svm import LinearSVC

# parameter C chosen experimentally (see explanation below)
C = 1.0

clf = LinearSVC(C=C)<jupyter_output><empty_output><jupyter_text>Training<jupyter_code># this may take some time
clf.fit(X_training, y_training)<jupyter_output><empty_output><jupyter_text>Testing and score We obtain **0.4914** score on CIFAR10 testing dataset.<jupyter_code>clf.score( X_testing, y_testing )<jupyter_output><empty_output><jupyter_text>Short look at the prediction<jupyter_code>y_predict = clf.predict( X_testing )
import numpy as np
np.unique( y_predict )<jupyter_output><empty_output><jupyter_text>Our results for different parametersTODO: * show misclassifications  Additional remarks Tuning C parameter<jupyter_code>for C in [ 0.001, 0.01, 0.1, 1.0, 1.2, 1.5, 2.0, 10.0 ]:
    clf = LinearSVC(C=C)
    clf.fit(X_training, y_training)
    print( 'normalize={norm}, C={C}, score={score}'.format(norm=normalize, C=C, score=clf.score( X_testing, y_testing )) )<jupyter_output>normalize=True, C=0.001, score=0.4758
normalize=True, C=0.01, score=0.4857
normalize=True, C=0.1, score=0.4904
normalize=True, C=1.0, score=0.4917
normalize=True, C=1.2, score=0.4908
normalize=True, C=1.5, score=0.4908
normalize=True, C=2.0, score=0.4908
normalize=True, C=10.0, score=0.4848<jupyter_text>SVC(kernel=linear) and SVC()<jupyter_code>from sklearn.svm import SVC
svc_lin_clf = SVC(kernel='linear', C=1)
svc_lin_clf.fit(X_training, y_training)
svc_lin_clf.score(X_testing, y_testing)
from sklearn.svm import SVC
svc_clf = SVC(C=1)
svc_clf.fit(X_training, y_training)
svc_clf.score(X_testing, y_testing)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13044903310390035, 'fraction_numerical': 0.027204195345788267, 'mean_word_length': 6.581366459627329, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '278314', 'license': 'MIT', 'n_tokens_mistral': 2368, 'n_tokens_neox': 2261, 'n_words': 443, 'path': 'Classification_using_HOG_features.ipynb', 'repo_name': 'rnoxy/cifar10-cnn', 'size': '6102'}