<jupyter_start><jupyter_text>Rock Paper and Scissors.In this notebook we are goingto create a `NN` that classifies between `rock` paper and `scissors` on images. The images are comming from Kaggle.* [Download](https://www.kaggle.com/drgfreeman/rockpaperscissors)<jupyter_code>import tensorflow as tf
import numpy as np
from tensorflow import keras
from tensorflow import image
from tensorflow.keras.preprocessing import image, image_dataset_from_directory
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>DataOur folder structure looks as follows:```data    rock    -img    paper    -img    scissors    -img```<jupyter_code>datagen = image.ImageDataGenerator(
    rescale=1.0 / 255,
    data_format="channels_last",
    dtype=tf.float32,
    validation_split=0.1,
)
train_ds = datagen.flow_from_directory(
    "data",
    target_size=(96, 96),
    batch_size=32,
    color_mode="grayscale",
    class_mode="sparse",
    shuffle=True,
    seed=42,
    subset="training",
)
valid_ds = datagen.flow_from_directory(
    "data",
    target_size=(96, 96),
    batch_size=32,
    color_mode="grayscale",
    class_mode="sparse",
    shuffle=True,
    seed=42,
    subset="validation",
)<jupyter_output>Found 1970 images belonging to 3 classes.
Found 218 images belonging to 3 classes.<jupyter_text>Class Names<jupyter_code>class_names = train_ds.class_indices
class_names
for batch in train_ds:
    break
plt.imshow(batch[0][0], cmap="gray")
plt.show()
batch[1][0]<jupyter_output><empty_output><jupyter_text>Creating a `NN`The `NN` achitecture will look as follows:```py        [ ConvBlock ] -> Conv2D -> BatchNormalization -> ReLU              |             |        [Flatten Layer / GlobalMaxPooling2D]             |        [Output Block] -> Dense Layers```* We need to improve this model achitecture in the future so that it will contain the following:    * Regularization / Dropout    * Pooling Layers e.g `MaxPooling2D`<jupyter_code>class ConvBlock(keras.layers.Layer):
    def __init__(self, in_features, kernel_size=(3,3)):
        super(ConvBlock, self).__init__()
        self.conv = keras.layers.Conv2D(in_features, kernel_size=kernel_size, padding="same")
        self.bn = keras.layers.BatchNormalization()
        self.relu = keras.activations.relu
        
    def call(self, x, training=False):
        x = self.conv(x)
        x = self.bn(x, training=training)
        return self.relu(x)
seq_model = keras.Sequential([
    keras.layers.Input(shape=(96, 96, 1)),
    ConvBlock(64),
    ConvBlock(128),
    ConvBlock(64),
    keras.layers.Flatten(),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(128, activation='relu'),
    keras.layers.Dense(64, activation='relu'),
    keras.layers.Dense(3, activation='softmax'),
], name="seq_model")
seq_model.summary()<jupyter_output>Model: "seq_model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv_block_12 (ConvBlock)    (None, 96, 96, 64)        896       
_________________________________________________________________
conv_block_13 (ConvBlock)    (None, 96, 96, 128)       74368     
_________________________________________________________________
conv_block_14 (ConvBlock)    (None, 96, 96, 64)        74048     
_________________________________________________________________
flatten_4 (Flatten)          (None, 589824)            0         
_________________________________________________________________
dense_16 (Dense)             (None, 64)                37748800  
_________________________________________________________________
dense_17 (Dense)             (None, 128)               8320      
_________________________________________________________[...]<jupyter_text>Creating Our Model using ``SubClassing`` API.<jupyter_code>class RPSModel(keras.Model):
    def __init__(self, in_features=np.array([64, 128, 64]), n_classes=3):
        super().__init__()
        self.conv_1 = ConvBlock(in_features[0])
        self.conv_2 = ConvBlock(in_features[1])
        self.conv_3 = ConvBlock(in_features[2])
        self.flatten = keras.layers.Flatten()
        self.dense_1 = keras.layers.Dense(in_features[0], activation='relu')
        self.dense_2 = keras.layers.Dense(in_features[1], activation='relu')
        self.dense_3 = keras.layers.Dense(in_features[2], activation='relu')
        self.dense_4 = keras.layers.Dense(n_classes, activation='softmax')
    
    def call(self, x, training=False):
        x = self.conv_1(x, training=training)
        x = self.conv_2(x, training=training)
        x = self.conv_3(x, training=training)
        x = self.flatten(x)
        x = self.dense_1(x)
        x = self.dense_2(x)
        x = self.dense_3(x)
        return self.dense_4(x)
    
    def model(self):
        x = keras.layers.Input((96, 96, 1))
        return keras.Model(inputs=[x], outputs=self.call(x))
rps_model = RPSModel()
rps_model.model().summary()<jupyter_output>Model: "model_8"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_7 (InputLayer)         [(None, 96, 96, 1)]       0         
_________________________________________________________________
conv_block_15 (ConvBlock)    (None, 96, 96, 64)        896       
_________________________________________________________________
conv_block_16 (ConvBlock)    (None, 96, 96, 128)       74368     
_________________________________________________________________
conv_block_17 (ConvBlock)    (None, 96, 96, 64)        74048     
_________________________________________________________________
flatten_5 (Flatten)          (None, 589824)            0         
_________________________________________________________________
dense_20 (Dense)             (None, 64)                37748800  
___________________________________________________________[...]<jupyter_text>Trainning the `Subclassing` Model.<jupyter_code>rps_model.compile(
    loss = keras.losses.SparseCategoricalCrossentropy(from_logits=False),
    optimizer = keras.optimizers.Adam(),
    metrics = ["acc"]
)

lr_reduction = keras.callbacks.ReduceLROnPlateau(
    monitor='val_acc', 
    patience=3, 
    verbose=1, 
    factor=0.5, 
    min_lr=0.00001
)

early_stopping = keras.callbacks.EarlyStopping(
    monitor='val_loss',
    min_delta=0,
    patience=5,
    verbose=0, mode='auto'
)

history = rps_model.fit(
    train_ds,
    batch_size=32,
    epochs =1,
    validation_data = train_ds,
    verbose = 1,
    shuffle=True,
    callbacks = [lr_reduction, early_stopping]
)<jupyter_output><empty_output><jupyter_text>Trainning the `Sequential` Model.<jupyter_code>seq_model.compile(
    loss = keras.losses.SparseCategoricalCrossentropy(from_logits=False),
    optimizer = keras.optimizers.Adam(),
    metrics = ["acc"]
)

lr_reduction = keras.callbacks.ReduceLROnPlateau(
    monitor='val_acc', 
    patience=3, 
    verbose=1, 
    factor=0.5, 
    min_lr=0.00001
)

early_stopping = keras.callbacks.EarlyStopping(
    monitor='val_loss',
    min_delta=0,
    patience=5,
    verbose=0, mode='auto'
)

history = seq_model.fit(
    train_ds,
    batch_size=32,
    epochs =5,
    validation_data = train_ds,
    verbose = 1,
    shuffle=True,
    callbacks = [lr_reduction, early_stopping]
)<jupyter_output><empty_output><jupyter_text>> The model above is not performing well on the test set. It is `overfitting` on the train data which is not what we want.**Solutions to overfitting problem:*** Dropout and Regulirazation* Maybe MaxPooling2D `Functional API` Model that performs better on the validation data.* [~99% accuracy model on Kaggle](https://www.kaggle.com/ritikmeena/rock-paper-scissor-99-accuracy)<jupyter_code>input_layer = keras.layers.Input(shape=(96,96,1), name="input_layer")

conv_1 = keras.layers.Conv2D(32, (3, 3), activation="relu")(input_layer)
bn_1 = keras.layers.BatchNormalization()(conv_1)
max_pool_2d1 = keras.layers.MaxPooling2D((2, 2))(conv_1)

conv_2 = keras.layers.Conv2D(32, (3, 3), activation="relu")(max_pool_2d1)
max_pool_2d2 = keras.layers.MaxPooling2D((2, 2))(conv_2)


flatten_layer = keras.layers.Flatten()(max_pool_2d2)
dense_1 = keras.layers.Dense(512, activation="relu")(flatten_layer)
output_layer = keras.layers.Dense(3, activation="softmax")(dense_1)

fn_model = keras.Model(inputs=input_layer, outputs=output_layer, name="fn_model")
fn_model.summary()
fn_model.compile(loss = 'sparse_categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
lr_reduction = keras.callbacks.ReduceLROnPlateau(
    monitor='val_accuracy', 
    patience=3, 
    verbose=1, 
    factor=0.5, 
    min_lr=0.00001
)

early_stopping = keras.callbacks.EarlyStopping(
    monitor='val_loss',
    min_delta=0,
    patience=5,
    verbose=0, mode='auto'
)

history = fn_model.fit(
    train_ds,
    batch_size=32,
    epochs =2,
    validation_data = valid_ds,
    verbose = 1,
    shuffle=True,
    callbacks = [lr_reduction, early_stopping]
)<jupyter_output>Epoch 1/2
62/62 [==============================] - 36s 571ms/step - loss: 1.0746 - accuracy: 0.4652 - val_loss: 0.4101 - val_accuracy: 0.8716
Epoch 2/2
62/62 [==============================] - 26s 412ms/step - loss: 0.3749 - accuracy: 0.8736 - val_loss: 0.2596 - val_accuracy: 0.9495<jupyter_text>> Now we are getting `~95%` accuracy with only 2 epochs.   Making `Predictions`<jupyter_code>for ds in valid_ds:
    break
predictions = np.argmax(model.predict(ds[0]), axis=1)
predictions, ds[1]
class_names
classes = list(class_names.keys())
classes
def plot_predictions_images(images_and_classes, labels_true, labels_pred, cols=5):
    rows = 3
    fig = plt.figure()
    fig.set_size_inches(cols * 2, rows * 2)
    for i, (image, label_true, label_pred) in enumerate(zip(images_and_classes, labels_true.astype("int32"), labels_pred)):
        plt.subplot(rows, cols, i + 1)
        plt.axis('off')
        plt.imshow(image, cmap="gray")
        plt.title(classes[label_true], color ='g' if label_true == label_pred else 'r', fontsize=16 )

plot_predictions_images(ds[0][:24], ds[1][:24], predictions[:24], cols=8)<jupyter_output><empty_output><jupyter_text>Confusion matrix<jupyter_code>import itertools
from sklearn.metrics import confusion_matrix
def plot_confusion_matrix(y_true, y_pred, classes=None, figsize=(5, 5), text_size=20): 
    cm = confusion_matrix(y_true, y_pred)
    cm_norm = cm.astype("float") / cm.sum(axis=1)[:, np.newaxis] 
    n_classes = cm.shape[0]
  
    fig, ax = plt.subplots(figsize=figsize)
    cax = ax.matshow(cm, cmap=plt.cm.Blues) 
    fig.colorbar(cax)

    if classes:
        labels = classes
    else:
        labels = np.arange(cm.shape[0])

    ax.set(title="Confusion Matrix",
         xlabel="Predicted label",
         ylabel="True label",
         xticks=np.arange(n_classes),
         yticks=np.arange(n_classes), 
         xticklabels=labels,
         yticklabels=labels,
          )
    ax.yaxis.label.set_color('green')
    ax.xaxis.label.set_color('green')

    
    ax.xaxis.set_label_position("bottom")
    ax.xaxis.tick_bottom()

    threshold = (cm.max() + cm.min()) / 2.
    # Plot the text on each cell
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, f"{cm[i, j]} ({cm_norm[i, j]*100:.1f}%)",
                 horizontalalignment="center",
                 color="white" if cm[i, j] > threshold else "black",
                 size=text_size)
plot_confusion_matrix(ds[1], predictions, classes=class_names.keys(), figsize=(8, 8))<jupyter_output><empty_output><jupyter_text>> With two epochs, this model is the best.  How to Create a custom `.compile()`, `.fit()` and `.evaluate()` > Creating our custom `.compile()`, `.fit()` and `.evaluate()`<jupyter_code>class Model(keras.Model):
    def __init__(self, model):
        super(Model, self).__init__()
        self.model = model
        
    # .compile
    def compile(self, loss, optimizer, metrics):
        super().compile()
        self.loss = loss
        self.optimizer = optimizer
        self.custom_metrics = metrics
    
    # .fit
    def train_step(self, data):
        x, y = data
        with tf.GradientTape() as tape:
            pred = self.model(x, training=True)
            loss = self.loss(y, pred)
        
        gradients = tape.gradient(zip(loss, self.trainable_variables))
        self.custom_metrics.update_state(y, pred)
        return {"loss": loss, "acc": self.custom_metrics.result()}
    
    # .evaluate
    
    def test_step(self, data):
        x, y = data
        pred = self.model(x, training=False)
        loss = self.loss(y, pred)
        self.custom_metrics.update_state(y, y_ped)
        return {"loss": loss, "acc": self.custom_metrics.result()}
    
    def call(self, inputs, training):
        self.model(inputs, training=training)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.13402932371228987, 'fraction_numerical': 0.033008367237276424, 'mean_word_length': 3.6930835734870318, 'pattern_counts': {'":': 4, '<': 41, '<?xml version=': 0, '>': 50, 'https://': 2, 'lorem ipsum': 0, 'www.': 2, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '11', 'dir': 'jupyter-structured-clean-dedup', 'id': '110758', 'license': 'MIT', 'n_tokens_mistral': 4545, 'n_tokens_neox': 4204, 'n_words': 825, 'path': '01_Classification/02_Rock_Paper_Scissors/01_Rock_Paper_Scissors.ipynb', 'repo_name': 'CrispenGari/Computer-Vision-In-TensorFlow', 'size': '13027'}