<jupyter_start><jupyter_text>Create a Siamese Network with Triplet Loss in Keras<jupyter_code>%matplotlib notebook

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import random

from pca_plotter import PCAPlotter

print('TensorFlow version:', tf.__version__)<jupyter_output>TensorFlow version: 2.2.0<jupyter_text>Understanding the ApproachWe have a CNN model called `EmbeddingModel`:We use three images for each training example:1. `person1_image1.jpg` (Anchor Example, represented below in green)2. `person1_image2.jpg` (Positive Example, in blue)3. `person2_image1.jpg` (Negative Example, in red). Siamese NetworkAll the three images of an example pass through the model, and we get the three Embeddings: One for the Anchor Example, one for the Positive Example, and one for the Negative Example.The three instances of the `EmbeddingModel` shown above are not different instances. It's the same, shared model instance - i.e. the parameters are shared, and are updated for all the three paths simultaneously.   Importing the Data<jupyter_code>(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
print(x_train.shape)
x_train = np.reshape(x_train, (x_train.shape[0], 784))/255.
x_test = np.reshape(x_test, (x_test.shape[0], 784))/255.
print(x_train.shape)<jupyter_output>(60000, 784)<jupyter_text>Plotting Examples<jupyter_code>def plot_triplets(examples):
    plt.figure(figsize=(6, 2))
    for i in range(3):
        plt.subplot(1, 3, 1 + i)
        plt.imshow(np.reshape(examples[i], (28, 28)), cmap='binary')
        plt.xticks([])
        plt.yticks([])
    plt.show()
plot_triplets([x_train[0], x_train[1], x_train[2]])<jupyter_output><empty_output><jupyter_text>Batch of Triplets<jupyter_code>def create_batch(batch_size=256):
    x_anchors = np.zeros((batch_size, 784))
    x_positives = np.zeros((batch_size, 784))
    x_negatives = np.zeros((batch_size, 784))
    
    for i in range(0, batch_size):
        # We need to find an anchor, a positive example and a negative example
        random_index = random.randint(0, x_train.shape[0] - 1)
        x_anchor = x_train[random_index]
        y = y_train[random_index]
        
        indices_for_pos = np.squeeze(np.where(y_train == y))
        indices_for_neg = np.squeeze(np.where(y_train != y))
        
        x_positive = x_train[indices_for_pos[random.randint(0, len(indices_for_pos) - 1)]]
        x_negative = x_train[indices_for_neg[random.randint(0, len(indices_for_neg) - 1)]]
        
        x_anchors[i] = x_anchor
        x_positives[i] = x_positive
        x_negatives[i] = x_negative
        
    return [x_anchors, x_positives, x_negatives]
examples = create_batch(1)
plot_triplets(examples)<jupyter_output><empty_output><jupyter_text>Embedding Model<jupyter_code>emb_size = 64

embedding_model = tf.keras.models.Sequential([
    tf.keras.layers.Dense(64, activation='relu', input_shape=(784,)),
    tf.keras.layers.Dense(emb_size, activation='sigmoid')
])

embedding_model.summary()
example = np.expand_dims(x_train[0], axis=0)
example_emb = embedding_model.predict(example)[0]

print(example_emb)<jupyter_output>[0.51874965 0.39473772 0.5346661  0.55529344 0.44400147 0.47922474
 0.48152474 0.5348377  0.52217984 0.57354164 0.56001306 0.48692915
 0.514538   0.62952435 0.51505876 0.60178787 0.48195043 0.59936213
 0.44829935 0.49371463 0.4504379  0.35041508 0.37888744 0.56067973
 0.45088562 0.47206002 0.47631165 0.4269706  0.48501524 0.539842
 0.46093985 0.52983975 0.45212978 0.5250372  0.5039889  0.46442044
 0.46010494 0.5158964  0.49904272 0.48373646 0.5668945  0.5080956
 0.40575737 0.41663694 0.6529609  0.52860874 0.53593045 0.5209337
 0.5596914  0.5917763  0.5553394  0.49491772 0.43750364 0.42531046
 0.4873131  0.3902648  0.41960552 0.52036244 0.47399956 0.5058956
 0.5096482  0.62306195 0.56582755 0.6105506 ]<jupyter_text>Siamese Network<jupyter_code>input_anchor = tf.keras.layers.Input(shape=(784,))
input_positive = tf.keras.layers.Input(shape=(784,))
input_negative = tf.keras.layers.Input(shape=(784,))

embedding_anchor = embedding_model(input_anchor)
embedding_positive = embedding_model(input_positive)
embedding_negative = embedding_model(input_negative)

output = tf.keras.layers.concatenate([embedding_anchor, embedding_positive, embedding_negative], axis=1)

net = tf.keras.models.Model([input_anchor, input_positive, input_negative], output)
net.summary()<jupyter_output>Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            [(None, 784)]        0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            [(None, 784)]        0                                            
__________________________________________________________________________________________________
input_3 (InputLayer)            [(None, 784)]        0                                            
__________________________________________________________________________________________________
sequential (Sequential)         (None, 64)           54400       input_1[0][0]                [...]<jupyter_text>Triplet LossA loss function that tries to pull the Embeddings of Anchor and Positive Examples closer, and tries to push the Embeddings of Anchor and Negative Examples away from each other.Root mean square difference between Anchor and Positive examples in a batch of N images is:$\begin{equation}d_p = \sqrt{\frac{\sum_{i=0}^{N-1}(f(a_i) - f(p_i))^2}{N}}\end{equation}$Root mean square difference between Anchor and Negative examples in a batch of N images is:$\begin{equation}d_n = \sqrt{\frac{\sum_{i=0}^{N-1}(f(a_i) - f(n_i))^2}{N}}\end{equation}$For each example, we want:$\begin{equation}d_p \leq d_n\end{equation}$Therefore,$\begin{equation}d_p - d_n \leq 0\end{equation}$This condition is quite easily satisfied during the training.We will make it non-trivial by adding a margin (alpha):$\begin{equation}d_p - d_n + \alpha \leq 0\end{equation}$Given the condition above, the Triplet Loss L is defined as:$\begin{equation}L = max(d_p - d_n + \alpha, 0)\end{equation}$<jupyter_code>alpha = 0.2

def triplet_loss(y_true, y_pred):
    anchor, positive, negative = y_pred[:,:emb_size], y_pred[:,emb_size:2*emb_size], y_pred[:,2*emb_size:]
    positive_dist = tf.reduce_mean(tf.square(anchor - positive), axis=1)
    negative_dist = tf.reduce_mean(tf.square(anchor - negative), axis=1)
    return tf.maximum(positive_dist - negative_dist + alpha, 0.)<jupyter_output><empty_output><jupyter_text>Data Generator<jupyter_code>def data_generator(batch_size=256):
    while True:
        x = create_batch(batch_size)
        y = np.zeros((batch_size, 3*emb_size))
        yield x, y<jupyter_output><empty_output><jupyter_text>Model Training<jupyter_code>batch_size = 2048
epochs = 10
steps_per_epoch = int(x_train.shape[0]/batch_size)

net.compile(loss=triplet_loss, optimizer='adam')

_ = net.fit(
    data_generator(batch_size),
    steps_per_epoch=steps_per_epoch,
    epochs=epochs, verbose=False,
    callbacks=[
        PCAPlotter(
            plt, embedding_model,
            x_test[:1000], y_test[:1000]
        )]
)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.1197652707388637, 'fraction_numerical': 0.09349159775940251, 'mean_word_length': 4.299646643109541, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 33, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 6, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '79757', 'license': 'MIT', 'n_tokens_mistral': 2929, 'n_tokens_neox': 2483, 'n_words': 531, 'path': 'Deep Learning using Tensorflow Keras/Siamese-Network-with-Triplet-Loss-in-Keras/Siamese Network with Triplet Loss.ipynb', 'repo_name': 'Storiesbyharshit/Deep-Learning', 'size': '7498'}