<jupyter_start><jupyter_text>Siamese Network RNNIn this little experiment, we are going to create an "Y model" based on MNIST model. The "Y model" has two inputs and one outputs. The Y-Network uses the same input twice, both on the left and right RNN branches. The network combines the results using concatenate layer.  import libraries<jupyter_code>import numpy as np
from keras.layers import Dense, Input, SimpleRNN
from keras.models import Model
from keras.datasets import mnist
from keras.layers.merge import concatenate
import matplotlib.pyplot as plt<jupyter_output>Using TensorFlow backend.<jupyter_text>load data<jupyter_code>(x_train,y_train), (x_test,y_test) = mnist.load_data()<jupyter_output><empty_output><jupyter_text>compute the number of labels<jupyter_code>num_labels = len(np.unique(y_train))<jupyter_output><empty_output><jupyter_text>reshape and normalize input images<jupyter_code>image_size = x_train.shape[1]
x_train = np.reshape(x_train,[-1,image_size,image_size])
x_test = np.reshape(x_test,[-1,image_size,image_size])
x_train = x_train.astype('float32')/255
x_test = x_test.astype('float32')/255<jupyter_output><empty_output><jupyter_text>visualize the data<jupyter_code>indexes = np.random.randint(0,x_train.shape[0], size=25)
images = x_train[indexes]
labels = y_train[indexes]

plt.figure(figsize=(5,5))
for i in range(len(indexes)):
    plt.subplot(5,5,i+1)
    image = images[i]
    plt.imshow(image,cmap='gray')
    plt.axis('off')

plt.show()
plt.close('all')<jupyter_output><empty_output><jupyter_text>network parameters<jupyter_code>input_shape = (image_size,image_size)
units = 256
dropout = 0.4
epochs = 20<jupyter_output><empty_output><jupyter_text>left branch of the network<jupyter_code>left_inputs = Input(shape=input_shape)
x = SimpleRNN(units=units,dropout=dropout)(left_inputs)<jupyter_output><empty_output><jupyter_text>right branch of the network<jupyter_code>right_inputs = Input(shape=input_shape)
y = SimpleRNN(units=units,dropout=dropout)(right_inputs)<jupyter_output><empty_output><jupyter_text>merge left and right branches<jupyter_code>z = concatenate([x,y])<jupyter_output><empty_output><jupyter_text>outputs<jupyter_code>outputs = Dense(num_labels,activation='softmax')(z)<jupyter_output><empty_output><jupyter_text>build the model<jupyter_code>model = Model([left_inputs,right_inputs],outputs=outputs)
model.summary()<jupyter_output>Model: "model_1"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_1 (InputLayer)            (None, 28, 28)       0                                            
__________________________________________________________________________________________________
input_2 (InputLayer)            (None, 28, 28)       0                                            
__________________________________________________________________________________________________
simple_rnn_1 (SimpleRNN)        (None, 256)          72960       input_1[0][0]                    
__________________________________________________________________________________________________
simple_rnn_2 (SimpleRNN)        (None, 256)          72960       input_2[0][0]              [...]<jupyter_text>compile and train the model<jupyter_code>model.compile(optimizer='sgd',loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit([x_train,x_train],y_train,validation_data=([x_test,x_test],y_test),
          epochs=20)<jupyter_output>Train on 60000 samples, validate on 10000 samples
Epoch 1/20
60000/60000 [==============================] - 37s 615us/step - loss: 0.5557 - accuracy: 0.8296 - val_loss: 0.2196 - val_accuracy: 0.9320
Epoch 2/20
60000/60000 [==============================] - 46s 760us/step - loss: 0.2801 - accuracy: 0.9138 - val_loss: 0.1449 - val_accuracy: 0.9560
Epoch 3/20
60000/60000 [==============================] - 49s 824us/step - loss: 0.2180 - accuracy: 0.9323 - val_loss: 0.1253 - val_accuracy: 0.9627
Epoch 4/20
60000/60000 [==============================] - 47s 790us/step - loss: 0.1826 - accuracy: 0.9434 - val_loss: 0.0948 - val_accuracy: 0.9700
Epoch 5/20
60000/60000 [==============================] - 52s 869us/step - loss: 0.1616 - accuracy: 0.9492 - val_loss: 0.0902 - val_accuracy: 0.9714
Epoch 6/20
60000/60000 [==============================] - 55s 924us/step - loss: 0.1484 - accuracy: 0.9535 - val_loss: 0.0839 - val_accuracy: 0.9730
Epoch 7/20
60000/60000 [==============================] [...]<jupyter_text>evaluate the model<jupyter_code>score = model.evaluate([x_test,x_test],y_test)
print("\nTest accuracy: %.1f%%" % (100*score[1]))<jupyter_output>10000/10000 [==============================] - 2s 156us/step

Test accuracy: 98.6%
{'fraction_non_alphanumeric': 0.1731786731786732, 'fraction_numerical': 0.06939356939356939, 'mean_word_length': 5.301282051282051, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 49, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '13', 'dir': 'jupyter-structured-clean-dedup', 'id': '132478', 'license': 'MIT', 'n_tokens_mistral': 1811, 'n_tokens_neox': 1558, 'n_words': 293, 'path': 'Artificial_Neural_Network/Siamese_Network/Siamese_RNN.ipynb', 'repo_name': 'AndryRafam/DEEP-LEARNING-Python3', 'size': '4914'}