<jupyter_start><jupyter_text>$Imports$----`Required Imports`---<jupyter_code>import keras_tuner as kt
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt<jupyter_output><empty_output><jupyter_text>$Data$---`Collecting the Fashion Mnist data and going through basic Preprocessing`---<jupyter_code>class_names = ["T_shirt","Trouser","Pullover","Dress","Coat","Sandal","Shirt","Sneaker","Bag","Ankle boot"]
(X_train_FMnist,Y_train_FMnist), (X_test_FMnist, Y_test_FMnist) = keras.datasets.fashion_mnist.load_data()
X_train_FMnist = X_train_FMnist / 255.0
X_test = X_test_FMnist / 255.0
X_train = X_train_FMnist[:40000]
X_valid = X_train_FMnist[40000:]
Y_train = Y_train_FMnist[:40000]
Y_valid = Y_train_FMnist[40000:]
X_valid.shape
np.bincount(Y_train_FMnist) # There is no class imbalance
def plot_img(i,data):
  plt.imshow(np.array(data[0][i]).reshape(28,28),cmap="gray")
  plt.title(class_names[data[1][i]])
  plt.show()
plot_img(5,[X_train,Y_train_FMnist])
plt.figure(figsize=(10,5))
n = 1
for i in range(80,90):
  plt.subplot(2,5,n)
  plt.axis("off")
  plt.imshow(np.array(X_train[i]).reshape(28,28),cmap="gray")
  plt.title(class_names[Y_train_FMnist[i]]) # Here 0s are black and the white(shades) are 1s(greater than 0)
  n+=1<jupyter_output><empty_output><jupyter_text>$BaseLine$---`A base small model, as a threshold for other models performance`---<jupyter_code>Baseline_model = keras.Sequential([
                             layers.Conv2D(64,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(padding="same"),
                             layers.Conv2D(128,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.Dense(64,activation="relu",kernel_initializer="he_normal"),
                             layers.Dropout(0.5),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Baseline_model.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Baseline_model.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=50,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)
print(f"Eval : {Baseline_model.evaluate(X_test,Y_test_FMnist)}")
baseline_stats = [0.30727022886276245, 0.890999972820282]<jupyter_output><empty_output><jupyter_text>$Models$---`Trying out different architectures and improving them`--- $Model_v1$<jupyter_code>Model_v1 = keras.Sequential([
                             layers.Conv2D(64,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(padding="same"),
                             layers.Conv2D(128,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.MaxPool2D(padding="same"),
                             layers.Conv2D(256,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.BatchNormalization(),
                             layers.Dense(128,activation="relu",kernel_initializer="he_normal"),
                             layers.Dropout(0.5),
                             layers.Dense(64,activation="relu",kernel_initializer="he_normal"),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v1.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v1.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)<jupyter_output>Epoch 1/100
1250/1250 [==============================] - 22s 16ms/step - loss: 0.6254 - accuracy: 0.7751 - val_loss: 0.5537 - val_accuracy: 0.8143
Epoch 2/100
1250/1250 [==============================] - 16s 13ms/step - loss: 0.3981 - accuracy: 0.8589 - val_loss: 0.4725 - val_accuracy: 0.8383
Epoch 3/100
1250/1250 [==============================] - 16s 13ms/step - loss: 0.3364 - accuracy: 0.8811 - val_loss: 0.2984 - val_accuracy: 0.8931
Epoch 4/100
1250/1250 [==============================] - 16s 13ms/step - loss: 0.3018 - accuracy: 0.8917 - val_loss: 0.3102 - val_accuracy: 0.8876
Epoch 5/100
1250/1250 [==============================] - 17s 14ms/step - loss: 0.2753 - accuracy: 0.9004 - val_loss: 0.3558 - val_accuracy: 0.8722
Epoch 6/100
1250/1250 [==============================] - 17s 14ms/step - loss: 0.2580 - accuracy: 0.9069 - val_loss: 0.2855 - val_accuracy: 0.9014
Epoch 7/100
1250/1250 [==============================] - 18s 14ms/step - loss: 0.2367 - accuracy: 0.9139 - val_loss: 0[...]<jupyter_text>There is a little bit of overfitting<jupyter_code>print(f"Eval : {Model_v1.evaluate(X_test,Y_test_FMnist)}")
Model_v1_stats = [0.29866519570350647, 0.902400016784668]<jupyter_output><empty_output><jupyter_text>This is better than the base model. $Model_v2$<jupyter_code>Model_v2 = keras.Sequential([
                             layers.Conv2D(64,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(),
                             layers.Conv2D(128,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.MaxPool2D(),
                             layers.Conv2D(128,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.Dense(64,activation="relu",kernel_initializer="he_normal"),
                             layers.Dropout(0.5),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v2.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v2.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)<jupyter_output>Epoch 1/100
1250/1250 [==============================] - 13s 10ms/step - loss: 0.7739 - accuracy: 0.7200 - val_loss: 0.4237 - val_accuracy: 0.8400
Epoch 2/100
1250/1250 [==============================] - 13s 10ms/step - loss: 0.4546 - accuracy: 0.8414 - val_loss: 0.3502 - val_accuracy: 0.8734
Epoch 3/100
1250/1250 [==============================] - 13s 11ms/step - loss: 0.3765 - accuracy: 0.8698 - val_loss: 0.3358 - val_accuracy: 0.8773
Epoch 4/100
1250/1250 [==============================] - 13s 11ms/step - loss: 0.3266 - accuracy: 0.8884 - val_loss: 0.3010 - val_accuracy: 0.8953
Epoch 5/100
1250/1250 [==============================] - 13s 11ms/step - loss: 0.2916 - accuracy: 0.9000 - val_loss: 0.2799 - val_accuracy: 0.8982
Epoch 6/100
1250/1250 [==============================] - 13s 10ms/step - loss: 0.2624 - accuracy: 0.9086 - val_loss: 0.2832 - val_accuracy: 0.8974
Epoch 7/100
1250/1250 [==============================] - 12s 10ms/step - loss: 0.2365 - accuracy: 0.9174 - val_loss: 0[...]<jupyter_text>This is again overfitting<jupyter_code>print(f"Eval : {Model_v2.evaluate(X_test,Y_test_FMnist)}")
Model_v2_stats = [0.30554068088531494, 0.9053000211715698]<jupyter_output><empty_output><jupyter_text>$Model_v3$<jupyter_code>Model_v3 = keras.Sequential([
                             layers.Conv2D(32,3,1,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(),
                             layers.Conv2D(64,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.Conv2D(64,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.BatchNormalization(),
                             layers.Dense(32,activation="relu",kernel_initializer="he_normal"),
                             layers.Dropout(0.5),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v3.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v3.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)
Model_v3_stats = Model_v3.evaluate(X_test,Y_test_FMnist)
print(f"Eval : {Model_v3_stats[0],Model_v3_stats[1]}")<jupyter_output>Eval : (0.367096871137619, 0.8708000183105469)<jupyter_text>That's bad than baseline $Model_v4$<jupyter_code>Model_v4 = keras.Sequential([
                             layers.Conv2D(32,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(),
                             layers.Conv2D(64,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.MaxPool2D(),
                             layers.Conv2D(64,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v4.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v4.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)
Model_v4_stats = Model_v4.evaluate(X_test,Y_test_FMnist)
print(f"Eval : {Model_v4_stats[0],Model_v4_stats[1]}")<jupyter_output>Epoch 1/100
1250/1250 [==============================] - 8s 6ms/step - loss: 0.6319 - accuracy: 0.7702 - val_loss: 0.4541 - val_accuracy: 0.8371
Epoch 2/100
1250/1250 [==============================] - 8s 6ms/step - loss: 0.4066 - accuracy: 0.8531 - val_loss: 0.3829 - val_accuracy: 0.8609
Epoch 3/100
1250/1250 [==============================] - 8s 7ms/step - loss: 0.3488 - accuracy: 0.8732 - val_loss: 0.3555 - val_accuracy: 0.8697
Epoch 4/100
1250/1250 [==============================] - 7s 6ms/step - loss: 0.3138 - accuracy: 0.8848 - val_loss: 0.3211 - val_accuracy: 0.8824
Epoch 5/100
1250/1250 [==============================] - 8s 7ms/step - loss: 0.2875 - accuracy: 0.8964 - val_loss: 0.2881 - val_accuracy: 0.8944
Epoch 6/100
1250/1250 [==============================] - 8s 7ms/step - loss: 0.2657 - accuracy: 0.9027 - val_loss: 0.3009 - val_accuracy: 0.8900
Epoch 7/100
1250/1250 [==============================] - 8s 7ms/step - loss: 0.2496 - accuracy: 0.9089 - val_loss: 0.2772 - val_ac[...]<jupyter_text>This have same results as Model_v1 and Model_v2 but this is much smaller. $Model_v5$<jupyter_code>Model_v5 = keras.Sequential([
                             layers.Conv2D(64,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(),
                             layers.Conv2D(128,3,1,padding="same",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v5.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v5.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)
Model_v5_stats = Model_v5.evaluate(X_test,Y_test_FMnist)
print(f"Eval : {Model_v5_stats[0],Model_v5_stats[1]}")<jupyter_output>1250/1250 [==============================] - 9s 7ms/step - loss: 0.3216 - accuracy: 0.8845 - val_loss: 0.3434 - val_accuracy: 0.8760
Epoch 12/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.3090 - accuracy: 0.8880 - val_loss: 0.3110 - val_accuracy: 0.8891
Epoch 13/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.3009 - accuracy: 0.8906 - val_loss: 0.3117 - val_accuracy: 0.8879
Epoch 14/100
1250/1250 [==============================] - 8s 6ms/step - loss: 0.2961 - accuracy: 0.8930 - val_loss: 0.3316 - val_accuracy: 0.8822
Epoch 15/100
1250/1250 [==============================] - 7s 6ms/step - loss: 0.2829 - accuracy: 0.8971 - val_loss: 0.3176 - val_accuracy: 0.8895
313/313 [==============================] - 1s 4ms/step - loss: 0.3403 - accuracy: 0.8805
Eval : (0.34027916193008423, 0.8805000185966492)<jupyter_text>Bad than the BaseLine $Model_v6$<jupyter_code>Model_v6 = keras.Sequential([
                             layers.Conv2D(64,5,2,padding="same",activation="relu",kernel_initializer="he_normal",input_shape=(28,28,1)),
                             layers.MaxPool2D(),
                             layers.Conv2D(64,3,1,padding="valid",activation="relu",kernel_initializer="he_normal"),
                             layers.Conv2D(128,3,1,padding="valid",activation="relu",kernel_initializer="he_normal"),
                             layers.GlobalAveragePooling2D(),
                             layers.Flatten(),
                             layers.BatchNormalization(),
                             layers.Dense(10,activation="softmax",kernel_initializer="glorot_uniform"),
])
Model_v6.compile(
    loss=keras.losses.sparse_categorical_crossentropy,
    optimizer="adam",
    metrics=["accuracy"]
)
Model_v6.fit(
    X_train,Y_train,
    validation_data=(X_valid,Y_valid),
    epochs=100,
    callbacks=[keras.callbacks.EarlyStopping(patience=3)]
)
Model_v6_stats = Model_v6.evaluate(X_test,Y_test_FMnist)
print(f"Eval : {Model_v6_stats[0],Model_v6_stats[1]}")<jupyter_output>Epoch 1/100
1250/1250 [==============================] - 11s 8ms/step - loss: 0.5114 - accuracy: 0.8132 - val_loss: 0.5065 - val_accuracy: 0.8192
Epoch 2/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.3486 - accuracy: 0.8734 - val_loss: 0.3618 - val_accuracy: 0.8745
Epoch 3/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.2964 - accuracy: 0.8895 - val_loss: 0.3994 - val_accuracy: 0.8537
Epoch 4/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.2697 - accuracy: 0.9025 - val_loss: 0.2829 - val_accuracy: 0.8967
Epoch 5/100
1250/1250 [==============================] - 10s 8ms/step - loss: 0.2431 - accuracy: 0.9099 - val_loss: 0.3252 - val_accuracy: 0.8890
Epoch 6/100
1250/1250 [==============================] - 9s 7ms/step - loss: 0.2217 - accuracy: 0.9187 - val_loss: 0.3015 - val_accuracy: 0.8956
Epoch 7/100
1250/1250 [==============================] - 10s 8ms/step - loss: 0.2041 - accuracy: 0.9250 - val_loss: 0.2943 - val[...]<jupyter_text>This is bad $Comparision$---`Choosing Final Model`---<jupyter_code>Loss_stats = [Model_v1_stats[0],Model_v2_stats[0], Model_v3_stats[0], Model_v4_stats[0], Model_v5_stats[0], Model_v6_stats[0]]
Acc_stats = [Model_v1_stats[1],Model_v2_stats[1], Model_v3_stats[1], Model_v4_stats[1], Model_v5_stats[1], Model_v6_stats[1]]
x_axis = [1,2,3,4,5,6]
idx = Loss_stats.index(min(Loss_stats)) + 1
plt.bar(x_axis,Loss_stats)
plt.bar(idx,min(Loss_stats),color="r")
plt.xlabel("Models")
plt.ylabel("Loss")
plt.show()
x_axis = [1,2,3,4,5,6]
idx = Acc_stats.index(max(Acc_stats)) + 1
plt.bar(x_axis,Acc_stats)
plt.bar(idx,max(Acc_stats),color="r")
plt.xlabel("Models")
plt.ylabel("Accuracy")
plt.show()
plt.figure(figsize=(8,5))
plt.subplot(211)
x_axis = [1,2,3,4,5,6]
idx = Loss_stats.index(min(Loss_stats)) + 1
plt.bar(x_axis,Loss_stats)
plt.bar(idx,min(Loss_stats),color="r")
plt.ylabel("Loss")
plt.subplot(212)
x_axis = [1,2,3,4,5,6]
idx = Acc_stats.index(max(Acc_stats)) + 1
plt.bar(x_axis,Acc_stats)
plt.bar(idx,max(Acc_stats),color="r")
plt.xlabel("Models")
plt.ylabel("Accuracy")
plt.show()<jupyter_output><empty_output><jupyter_text>From this we can conclude that Model_v4 is the most robust/best model for Fashion Mnist. $Predicting$---`Using the best model so far to predict the classes`---<jupyter_code>def predictions(id):
  y_pred = np.argmax(Model_v4.predict(np.array(X_test[id]).reshape(1,28,28)))
  y_target = Y_test_FMnist[id]
  return y_pred, y_target
predictions(5) # Predicted value and Original Value
for i in range(45,78):
  y_pred, original = predictions(i)
  print(f"Index : {i}")
  print(f"Predicted : {y_pred} Original : {original}")
  if y_pred != original:
    print(f"Wrong Prediction, Index : {i}")<jupyter_output>Index : 45
Predicted : 7 Original : 7
Index : 46
Predicted : 2 Original : 2
Index : 47
Predicted : 1 Original : 1
Index : 48
Predicted : 2 Original : 2
Index : 49
Predicted : 6 Original : 2
Wrong Prediction, Index : 49
Index : 50
Predicted : 4 Original : 4
Index : 51
Predicted : 4 Original : 4
Index : 52
Predicted : 5 Original : 5
Index : 53
Predicted : 8 Original : 8
Index : 54
Predicted : 2 Original : 2
Index : 55
Predicted : 2 Original : 2
Index : 56
Predicted : 8 Original : 8
Index : 57
Predicted : 4 Original : 4
Index : 58
Predicted : 8 Original : 8
Index : 59
Predicted : 0 Original : 0
Index : 60
Predicted : 7 Original : 7
Index : 61
Predicted : 7 Original : 7
Index : 62
Predicted : 8 Original : 8
Index : 63
Predicted : 5 Original : 5
Index : 64
Predicted : 1 Original : 1
Index : 65
Predicted : 1 Original : 1
Index : 66
Predicted : 3 Original : 2
Wrong Prediction, Index : 66
Index : 67
Predicted : 3 Original : 3
Index : 68
Predicted : 7 Original : 9
Wrong Prediction, Index : 68
I[...]<jupyter_text>49, 66 and 68 index were wrong predictions, lets look at them.<jupyter_code>wrong_idx = [49,66,68]
for i in wrong_idx:
  plot_img(i,[X_test,Y_test_FMnist])<jupyter_output><empty_output><jupyter_text>I guess model was not able to grasp patterns for the class "Pullovers"<jupyter_code>def Model_predictions(start,end,plot):
  wrong_idx = []
  for i in range(start,end):
    y_pred, original = predictions(i)
    if y_pred != original:
      print(f"Wrong Prediction!!! Index : {i} \n Predicted : {class_names[y_pred]} & Original = {class_names[original]} ")
      wrong_idx.append(i)
  if plot:
    for i in wrong_idx:
      plot_img(i,[X_test,Y_test_FMnist])
Model_predictions(102,150,False)<jupyter_output>Wrong Prediction!!! Index : 103 
 Predicted : Shirt & Original = Pullover 
Wrong Prediction!!! Index : 107 
 Predicted : Sneaker & Original = Ankle boot 
Wrong Prediction!!! Index : 136 
 Predicted : Shirt & Original = Pullover 
Wrong Prediction!!! Index : 147 
 Predicted : Dress & Original = Shirt<jupyter_text>Yeah, my assumption was true. The Model is not able to learn patterns for the pullover class.<jupyter_code>Model_predictions(865,912,True)<jupyter_output><empty_output><jupyter_text>$Decision$ $Params$---`Using more methods to compare the model like precision and recall`---<jupyter_code>from sklearn.metrics import f1_score, confusion_matrix, precision_score, recall_score
Models = [Model_v1,Model_v2,Model_v3,Model_v4,Model_v5,Model_v6]
Y_pred = []
for i in range(6):
  Y_pred.append(np.argmax(Models[i].predict(X_test),axis=1))
Precisions = []
for y_pred in Y_pred:
  Precisions.append(precision_score(Y_test_FMnist,y_pred,average='macro'))

Recall = []
for y_pred in Y_pred:
  Recall.append(recall_score(Y_test_FMnist,y_pred,average='macro'))

CM = []
for y_pred in Y_pred:
  CM.append(confusion_matrix(Y_test_FMnist,y_pred))

F1 = []
for y_pred in Y_pred:
  F1.append(f1_score(Y_test_FMnist,y_pred,average='macro'))
x_axis = [1,2,3,4,5,6]
f, (ax1,ax2,ax3) = plt.subplots(1,3,sharex=True,sharey=True,)

ax1.bar(x_axis,Precisions)
idx = Precisions.index(max(Precisions)) +1
ax1.bar(idx,max(Precisions))
ax1.set_title("Precision")

ax2.bar(x_axis,Recall)
idx = Recall.index(max(Recall)) +1
ax2.bar(idx,max(Recall))
ax2.set_title("Recall")

ax3.bar(x_axis,F1)
idx = F1.index(max(F1)) +1
ax3.bar(idx,max(F1))
ax3.set_title("F1")

f.set_figheight(5)
f.set_figwidth(8)<jupyter_output><empty_output><jupyter_text>According to this Model 2 is better than Model 4. Lets find out.<jupyter_code># for i in range(6):
#   print(f"Model-{i+1}")
#   print(f"CM : {CM[i]}")

# # Only Run if yoy wanna look at them|.
def predictions_v2(id):
  y_pred = np.argmax(Model_v2.predict(np.array(X_test[id]).reshape(1,28,28)))
  y_target = Y_test_FMnist[id]
  return y_pred, y_target
def Model_predictions_v2(start,end):
  wrong_counter = 0
  for i in range(start,end):
    y_pred, original = predictions_v2(i)
    if y_pred != original:
      wrong_counter+=1
  return wrong_counter

def predictions_v4(id):
  y_pred = np.argmax(Model_v4.predict(np.array(X_test[id]).reshape(1,28,28)))
  y_target = Y_test_FMnist[id]
  return y_pred, y_target
def Model_predictions_v4(start,end):
  wrong_counter = 0
  for i in range(start,end):
    y_pred, original = predictions_v4(i)
    if y_pred != original:
      wrong_counter+=1
  return wrong_counter<jupyter_output><empty_output><jupyter_text>The idea is simple, with a step size of 10. We will check that in 10 steps which model made wrong predictions greater than the other.<jupyter_code>V2_wrongs = []
V4_wrongs = []
k = 100
for i in range(10):
  for j in range(1):
    n_wrong_v2 = Model_predictions_v2(k,k+100)
    n_wrong_v4 = Model_predictions_v4(k,k+100)
    V2_wrongs.append(n_wrong_v2)
    V4_wrongs.append(n_wrong_v4)
    print(f"K : {k,k+100} Wrong V2 : {n_wrong_v2} & Wrong v4 : {n_wrong_v4} ")
    k+=100
print(f"V2 Errors : {sum(V2_wrongs)}")
print(f"V4 Errors : {sum(V4_wrongs)}")<jupyter_output>K : (100, 200) Wrong V2 : 7 & Wrong v4 : 7 
K : (200, 300) Wrong V2 : 7 & Wrong v4 : 7 
K : (300, 400) Wrong V2 : 12 & Wrong v4 : 9 
K : (400, 500) Wrong V2 : 9 & Wrong v4 : 11 
K : (500, 600) Wrong V2 : 10 & Wrong v4 : 10 
K : (600, 700) Wrong V2 : 8 & Wrong v4 : 9 
K : (700, 800) Wrong V2 : 9 & Wrong v4 : 7 
K : (800, 900) Wrong V2 : 7 & Wrong v4 : 7 
K : (900, 1000) Wrong V2 : 15 & Wrong v4 : 16 
K : (1000, 1100) Wrong V2 : 4 & Wrong v4 : 8 
V2 Errors : 88
V4 Errors : 91<jupyter_text>You can check it for different steps or for different range. In general, that's true Model V2 is better than Model V4.<jupyter_code>print(f"V2 Acc : {Model_v2_stats[1]}")
print(f"V4 Acc : {Model_v4_stats[1]}")<jupyter_output>V2 Acc : 0.9053000211715698
V4 Acc : 0.9046000242233276
{'fraction_non_alphanumeric': 0.1772800981724153, 'fraction_numerical': 0.09256256300127098, 'mean_word_length': 3.7587069864442126, 'pattern_counts': {'":': 0, '<': 71, '<?xml version=': 0, '>': 71, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '20', 'dir': 'jupyter-structured-clean-dedup', 'id': '516909', 'license': 'Apache-2.0', 'n_tokens_mistral': 9687, 'n_tokens_neox': 8260, 'n_words': 1450, 'path': 'CNN_1(Fashion_MNIST).ipynb', 'repo_name': 'Coderyug2021/Step-To-Computer-Vision', 'size': '22817'}