<jupyter_start><jupyter_text>Cifar10 Neural Nets vs classic algorithmsOn this notebook we are going to compare neural networks performanc against the classic algorithms on the same task. The dataset:As we mentoioned, we are going to use the CIFAR10 data. Let's take a look:The dataset is composed by 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. These are the image classes:- airplane - automobile - bird- cat- deer- dog- frog- horse- ship- truck<jupyter_code>import tensorflow as tf

physical_devices = tf.config.experimental.list_physical_devices('GPU')
tf.config.experimental.set_memory_growth(physical_devices[0], True)

tf.keras.backend.clear_session()  # For easy reset of notebook state.

from tensorflow import keras
from tensorflow.keras.datasets import cifar10

num_classes = 10
# The data, split between train and test sets:
(trainX, y_train), (testX, y_test) = cifar10.load_data()
x_train = trainX.reshape(trainX.shape[0],-1)
x_test = testX.reshape(testX.shape[0], -1)

print('x_train shape:', x_train.shape)
print(x_train.shape[0], 'train samples')
print(x_test.shape[0], 'test samples')

# Convert class vectors to binary class matrices.
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)<jupyter_output>x_train shape: (50000, 3072)
50000 train samples
10000 test samples<jupyter_text>Plot some examples of the dataset.Hint: use the imshow function of the pyplot package<jupyter_code>from matplotlib import pyplot as plt
%matplotlib inline
...<jupyter_output><empty_output><jupyter_text>Classic approachOne of the main takeouts from DL is that it extract features from the data. We are facing images. So, let's reduce the disadvantage of the classic algorithms and first, let's get some interesting features. We will do a feature extraction using pca, and then will test against some algorithms like random forest, svm, knn or logistic regression PCA<jupyter_code>import numpy as np
from sklearn.metrics import accuracy_score<jupyter_output><empty_output><jupyter_text>Perform the PCA fitting to the data and extract the optimal number of components to explain at least 95% of the variance explainedHint: find the appropiated attribute on the pca<jupyter_code>from sklearn.decomposition import PCA
...<jupyter_output><empty_output><jupyter_text>Now create the pca with the given number of components. Fit the training and transform the testing data.Hint: Use whiten=True<jupyter_code>## Applying PCA with k calcuated above
...<jupyter_output><empty_output><jupyter_text>Train and evaluate the random forestHint: What data are you using?<jupyter_code>from sklearn.ensemble import RandomForestClassifier
...
random_forest_score<jupyter_output><empty_output><jupyter_text>Train and evaluate the knnHint: What data are you using?<jupyter_code>from sklearn.neighbors import KNeighborsClassifier
...
knn_score<jupyter_output><empty_output><jupyter_text>Train and evaluate the logistic regressionHint: What data are you using?<jupyter_code>from sklearn.linear_model import LogisticRegression
...
logistic_regression_score<jupyter_output><empty_output><jupyter_text>Train and evaluate the svmHint: What data are you using?    Hint: Don't do it. It takes forever.    Hint: Really, please don't. It doesn't improve that much.<jupyter_code>from sklearn import svm
## Training
svc = svm.SVC()
...
svc_score<jupyter_output><empty_output><jupyter_text>Results:<jupyter_code>print("RandomForestClassifier : ", random_forest_score)
print("K Nearest Neighbors : ", knn_score)
print("Logistic Regression : ", logistic_regression_score)
# print("Support Vector Classifier : ", svc_score) # 0.4838<jupyter_output>RandomForestClassifier :  0.0048
K Nearest Neighbors :  0.106
Logistic Regression :  0.402
Support Vector Classifier :  0.4838
{'fraction_non_alphanumeric': 0.0784313725490196, 'fraction_numerical': 0.02218782249742002, 'mean_word_length': 6.328922495274102, 'pattern_counts': {'":': 0, '<': 39, '<?xml version=': 0, '>': 39, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '289703', 'license': 'Apache-2.0', 'n_tokens_mistral': 1288, 'n_tokens_neox': 1190, 'n_words': 397, 'path': 'Training/nnetsVsClassicCifarTraining.ipynb', 'repo_name': 'fermaat/afi_deep_learning_intro', 'size': '3876'}