<jupyter_start><jupyter_text>Activity 5.01: OLS Classifier - 2 Class Classifier (with Answers)<jupyter_code>import struct
import numpy as np
import gzip
import urllib.request
import matplotlib.pyplot as plt
from array import array
from sklearn.linear_model import LinearRegression<jupyter_output><empty_output><jupyter_text>Load the MNIST data into memory<jupyter_code>with gzip.open('../Datasets/train-images-idx3-ubyte.gz', 'rb') as f:
    magic, size, rows, cols = struct.unpack(">IIII", f.read(16))

    img = np.array(array("B", f.read())).reshape((size, rows, cols))
with gzip.open('../Datasets/train-labels-idx1-ubyte.gz', 'rb') as f:
    magic, size = struct.unpack(">II", f.read(8))
    labels = np.array(array("B", f.read()))
with gzip.open('../Datasets/t10k-images-idx3-ubyte.gz', 'rb') as f:
    magic, size, rows, cols = struct.unpack(">IIII", f.read(16))

    img_test = np.array(array("B", f.read())).reshape((size, rows, cols))
with gzip.open('../Datasets/t10k-labels-idx1-ubyte.gz', 'rb') as f:
    magic, size = struct.unpack(">II", f.read(8))
    labels_test = np.array(array("B", f.read()))<jupyter_output><empty_output><jupyter_text>Visualise a sample of the data<jupyter_code>for i in range(10):
    plt.subplot(2, 5, i + 1)
    plt.imshow(img[i], cmap='gray');
    plt.title(f'{labels[i]}');
    plt.axis('off')<jupyter_output><empty_output><jupyter_text>Construct a Linear Classifier Model to Classify Digits 0 / 1The first model we are going to create is to determine if the samples are either the digits 0 or 1.  To do this we first need to select only those samples.<jupyter_code>samples_0_1 = np.where((labels == 0) | (labels == 1))[0]
images_0_1 = img[samples_0_1]
labels_0_1 = labels[samples_0_1]

samples_0_1_test = np.where((labels_test == 0) | (labels_test == 1))
images_0_1_test = img_test[samples_0_1_test].reshape((-1, rows * cols))
labels_0_1_test = labels_test[samples_0_1_test]<jupyter_output><empty_output><jupyter_text>Visualising the selected information:<jupyter_code>sample_0 = np.where((labels == 0))[0][0]
plt.imshow(img[sample_0], cmap='gray');
sample_1 = np.where((labels == 1))[0][0]
plt.imshow(img[sample_1], cmap='gray');<jupyter_output><empty_output><jupyter_text>In order to provide the image information to the Logistic model we must first flatten the data out so that each image is 1 x 784 pixels in shape.<jupyter_code>images_0_1 = images_0_1.reshape((-1, rows * cols))
images_0_1.shape<jupyter_output><empty_output><jupyter_text>Let's construct the model, use the sklearn LogisticRegression API and call the fit function.<jupyter_code>model = LinearRegression()
model.fit(X=images_0_1, y=labels_0_1)<jupyter_output>/Users/ashish.jha/opt/anaconda3/lib/python3.7/site-packages/sklearn/linear_model/base.py:503: RuntimeWarning: internal gelsd driver lwork query error, required iwork dimension not returned. This is likely the result of LAPACK bug 0038, fixed in LAPACK 3.2.2 (released July 21, 2010). Falling back to 'gelss' driver.
  linalg.lstsq(X, y)<jupyter_text>Determine the training set accuracy<jupyter_code>model.score(X=images_0_1, y=labels_0_1)<jupyter_output><empty_output><jupyter_text>Determine the label predictions for each of the training samples, using a threshold of 0.5.  Values greater than 0.5 classify as 1, less than or equal to classify as 0.<jupyter_code>y_pred = model.predict(images_0_1) > 0.5
y_pred = y_pred.astype(int)
y_pred<jupyter_output><empty_output><jupyter_text>Compute the classification accuracy of the predicted training values vs the ground truth<jupyter_code>np.sum(y_pred == labels_0_1) / len(labels_0_1)<jupyter_output><empty_output><jupyter_text>Compare the performance against the test set<jupyter_code>y_pred = model.predict(images_0_1_test) > 0.5
y_pred = y_pred.astype(int)
np.sum(y_pred == labels_0_1_test) / len(labels_0_1_test)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13247089262613196, 'fraction_numerical': 0.031824062095730916, 'mean_word_length': 6.185873605947956, 'pattern_counts': {'":': 0, '<': 44, '<?xml version=': 0, '>': 50, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '331365', 'license': 'MIT', 'n_tokens_mistral': 1486, 'n_tokens_neox': 1425, 'n_words': 327, 'path': 'Chapter05/Activity5.01/Activity5.01 - OLS Classifier - Binary Classifier (with Answers).ipynb', 'repo_name': 'nijinjose/The-Supervised-Learning-Workshop', 'size': '3865'}