<jupyter_start><jupyter_text>* We'll start simple, with a single fully-connected neural layer as encoder and as decoder:<jupyter_code>from keras.layers import Input, Dense
from keras.models import Model

# this is the size of our encoded representations
encoding_dim = 32  # 32 floats -> compression of factor 24.5, assuming the input is 784 floats

# this is our input placeholder
input_img = Input(shape=(784,))
# "encoded" is the encoded representation of the input
encoded = Dense(encoding_dim, activation='relu')(input_img)
# "decoded" is the lossy reconstruction of the input
decoded = Dense(784, activation='sigmoid')(encoded)

# this model maps an input to its reconstruction
autoencoder = Model(input_img, decoded)<jupyter_output><empty_output><jupyter_text>* Let's also create a separate encoder model:<jupyter_code># this model maps an input to its encoded representation
encoder = Model(input_img, encoded)<jupyter_output><empty_output><jupyter_text>* As well as the decoder model:<jupyter_code># create a placeholder for an encoded (32-dimensional) input
encoded_input = Input(shape=(encoding_dim,))
# retrieve the last layer of the autoencoder model
decoder_layer = autoencoder.layers[-1]
# create the decoder model
decoder = Model(encoded_input, decoder_layer(encoded_input))<jupyter_output><empty_output><jupyter_text>* Now let's train our autoencoder to reconstruct MNIST digits.* First, we'll configure our model to use a per-pixel binary crossentropy loss, and the Adadelta optimizer:<jupyter_code>autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')<jupyter_output><empty_output><jupyter_text>* Let's prepare our input data. We're using MNIST digits, and we're discarding the labels (since we're only interested in encoding/decoding the input images).<jupyter_code>from keras.datasets import mnist
import numpy as np
(x_train, _), (x_test, _) = mnist.load_data()<jupyter_output><empty_output><jupyter_text>* We will normalize all values between 0 and 1 and we will flatten the 28x28 images into vectors of size 784.<jupyter_code>x_train = x_train.astype('float32') / 255.
x_test = x_test.astype('float32') / 255.
x_train = x_train.reshape((len(x_train), np.prod(x_train.shape[1:])))
x_test = x_test.reshape((len(x_test), np.prod(x_test.shape[1:])))
print (x_train.shape)
print (x_test.shape)<jupyter_output>(60000, 784)
(10000, 784)<jupyter_text>* Now let's train our autoencoder for 50 epochs:<jupyter_code>autoencoder.fit(x_train, x_train,
                epochs=50,
                batch_size=256,
                shuffle=True,
                validation_data=(x_test, x_test))<jupyter_output>Train on 60000 samples, validate on 10000 samples
Epoch 1/50
60000/60000 [==============================] - 2s 41us/step - loss: 0.3641 - val_loss: 0.2708
Epoch 2/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.2627 - val_loss: 0.2507
Epoch 3/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.2401 - val_loss: 0.2276
Epoch 4/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.2202 - val_loss: 0.2106
Epoch 5/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.2056 - val_loss: 0.1981
Epoch 6/50
60000/60000 [==============================] - 2s 36us/step - loss: 0.1949 - val_loss: 0.1890
Epoch 7/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.1869 - val_loss: 0.1819
Epoch 8/50
60000/60000 [==============================] - 2s 34us/step - loss: 0.1805 - val_loss: 0.1761
Epoch 9/50
60000/60000 [==============================] - 2s 35us/step - loss: 0.1749 - val_loss: 0.1709
Epoch[...]<jupyter_text>* After 50 epochs, the autoencoder seems to reach a stable train/test loss value of about 0.11. We can try to visualize the reconstructed inputs and the encoded representations. We will use Matplotlib.<jupyter_code># encode and decode some digits
# note that we take them from the *test* set
encoded_imgs = encoder.predict(x_test)
decoded_imgs = decoder.predict(encoded_imgs)
# use Matplotlib (don't ask)
import matplotlib.pyplot as plt

n = 10  # how many digits we will display
plt.figure(figsize=(20, 4))
for i in range(n):
    # display original
    ax = plt.subplot(2, n, i + 1)
    plt.imshow(x_test[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)

    # display reconstruction
    ax = plt.subplot(2, n, i + 1 + n)
    plt.imshow(decoded_imgs[i].reshape(28, 28))
    plt.gray()
    ax.get_xaxis().set_visible(False)
    ax.get_yaxis().set_visible(False)
plt.show()<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.16421143847487002, 'fraction_numerical': 0.0712738301559792, 'mean_word_length': 5.043193717277487, 'pattern_counts': {'":': 0, '<': 31, '<?xml version=': 0, '>': 32, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '8', 'dir': 'jupyter-structured-clean-dedup', 'id': '504084', 'license': 'MIT', 'n_tokens_mistral': 1818, 'n_tokens_neox': 1531, 'n_words': 436, 'path': 'AutoEncoder1.ipynb', 'repo_name': 'Abdel-Moussaoui/GAN-Master', 'size': '4616'}