<jupyter_start><jupyter_text>Testing the architectures designed The goal of this notebook is to evaluate the architectures designed according as has been explained in the methodology section. To do so, the same workbench will be used to train and evaluate them under the same circumstances. As explained in the report, each architecture will be trained using both MSE and SSIM loss functions.  (Previous preparation)<jupyter_code>import os
import sys
module_path = os.path.abspath(os.path.join('..'))
if module_path not in sys.path:
    sys.path.append(module_path)<jupyter_output><empty_output><jupyter_text>Importing packages<jupyter_code>from src.helpers import load_experiment_workbench, use_gpu
from src.models import AutoEncoder
from src.models.autoencoder.modules.encoders.models import SimpleEncoder, ResnetEncoderv1, ResnetEncoderv2
from src.trainer.metrics import psnr, ssim
from src.trainer.losses import ssim as ssim_loss

from tensorflow.keras import layers<jupyter_output><empty_output><jupyter_text>Enabling GPU<jupyter_code>use_gpu(True)<jupyter_output>1 Physical GPUs, 1 Logical GPUs<jupyter_text>Creating a new workbench<jupyter_code>workbench = load_experiment_workbench("architectures_desgined_8fm")<jupyter_output>Found 26305 validated image filenames.
Found 26305 validated image filenames.
Found 5573 validated image filenames.
Found 5573 validated image filenames.
Found 8039 validated image filenames.
Found 8039 validated image filenames.<jupyter_text>Define metrics<jupyter_code>metrics = [
    "mean_squared_error",
    psnr,
    ssim
]<jupyter_output><empty_output><jupyter_text>Architectures  AutoEncoder (AE)  Loss function: MSE<jupyter_code>ae_mse_model = AutoEncoder(
    encoder=SimpleEncoder(depth=4, feature_maps=8),
    name="ae_mse",
)(input_shape=workbench.image_shape)
ae_mse_model.compile(loss='mean_squared_error', optimizer = 'adam', metrics=metrics)
ae_mse_fitted = workbench.run(model=ae_mse_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 161s 98ms/step - loss: 0.0160 - mean_squared_error: 0.0160 - psnr: 19.9810 - ssim: 0.3544 - val_loss: 0.0054 - val_mean_squared_error: 0.0054 - val_psnr: 22.8859 - val_ssim: 0.4498
Epoch 2/30
1644/1644 [==============================] - 159s 97ms/step - loss: 0.0059 - mean_squared_error: 0.0059 - psnr: 22.7375 - ssim: 0.4915 - val_loss: 0.0054 - val_mean_squared_error: 0.0054 - val_psnr: 23.0791 - val_ssim: 0.5211
Epoch 3/30
1644/1644 [==============================] - 159s 97ms/step - loss: 0.0043 - mean_squared_error: 0.0043 - psnr: 23.9932 - ssim: 0.6108 - val_loss: 0.0038 - val_mean_squared_error: 0.0038 - val_psnr: 24.4565 - val_ssim: 0.6518
Epoch 4/30
1644/1644 [==============================] - 159s 97ms/step - loss: 0[...]<jupyter_text>Loss function: SSIM<jupyter_code>ae_ssim_model = AutoEncoder(
    encoder=SimpleEncoder(depth=4, feature_maps=8),
    name="ae_ssim",
)(input_shape=workbench.image_shape)
ae_ssim_model.compile(loss=ssim_loss, optimizer = 'adam', metrics=metrics)
ae_ssim_fitted = workbench.run(model=ae_ssim_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 174s 106ms/step - loss: 0.2363 - mean_squared_error: 0.0146 - psnr: 21.9347 - ssim: 0.7637 - val_loss: 0.2012 - val_mean_squared_error: 0.0057 - val_psnr: 22.8498 - val_ssim: 0.7988
Epoch 2/30
1644/1644 [==============================] - 172s 104ms/step - loss: 0.1559 - mean_squared_error: 0.0033 - psnr: 25.3075 - ssim: 0.8441 - val_loss: 0.1745 - val_mean_squared_error: 0.0037 - val_psnr: 25.0143 - val_ssim: 0.8256
Epoch 3/30
1644/1644 [==============================] - 172s 105ms/step - loss: 0.1421 - mean_squared_error: 0.0030 - psnr: 25.7063 - ssim: 0.8579 - val_loss: 0.1845 - val_mean_squared_error: 0.0044 - val_psnr: 24.7974 - val_ssim: 0.8155
Epoch 4/30
1644/1644 [==============================] - 172s 105ms/step - los[...]<jupyter_text>AE + Skippend Long Connections (AE+sLC)  Loss function: MSE<jupyter_code>ae_slc_mse_model = AutoEncoder(
    encoder=SimpleEncoder(depth=4, feature_maps=8),
    name="ae_slc_mse",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_mse_model.compile(loss='mean_squared_error', optimizer = 'adam', metrics=metrics)
ae_slc_mse_fitted = workbench.run(model=ae_slc_mse_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 165s 100ms/step - loss: 0.0060 - mean_squared_error: 0.0060 - psnr: 26.0533 - ssim: 0.6451 - val_loss: 0.0013 - val_mean_squared_error: 0.0013 - val_psnr: 30.3027 - val_ssim: 0.7882
Epoch 2/30
1644/1644 [==============================] - 164s 100ms/step - loss: 0.0013 - mean_squared_error: 0.0013 - psnr: 30.3893 - ssim: 0.8032 - val_loss: 9.6850e-04 - val_mean_squared_error: 9.6864e-04 - val_psnr: 32.5014 - val_ssim: 0.8790
Epoch 3/30
1644/1644 [==============================] - 164s 100ms/step - loss: 0.0011 - mean_squared_error: 0.0011 - psnr: 31.5362 - ssim: 0.8658 - val_loss: 9.1895e-04 - val_mean_squared_error: 9.1794e-04 - val_psnr: 32.2905 - val_ssim: 0.9165
Epoch 4/30
1643/1644 [============================>.] - ETA: [...]<jupyter_text>Loss function: SSIM<jupyter_code>ae_slc_ssim_model = AutoEncoder(
    encoder=SimpleEncoder(depth=4, feature_maps=8),
    name="ae_slc_ssim",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_ssim_model.compile(loss=ssim_loss, optimizer = 'adam', metrics=metrics)
ae_slc_ssim_fitted = workbench.run(model=ae_slc_ssim_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 178s 108ms/step - loss: 0.0969 - mean_squared_error: 0.0250 - psnr: 24.5536 - ssim: 0.9031 - val_loss: 0.0884 - val_mean_squared_error: 0.0162 - val_psnr: 23.3063 - val_ssim: 0.9116
Epoch 2/30
1644/1644 [==============================] - 177s 107ms/step - loss: 0.0408 - mean_squared_error: 0.0024 - psnr: 31.2117 - ssim: 0.9592 - val_loss: 0.0376 - val_mean_squared_error: 0.0011 - val_psnr: 32.0829 - val_ssim: 0.9624
Epoch 3/30
1644/1644 [==============================] - 177s 107ms/step - loss: 0.0335 - mean_squared_error: 9.5328e-04 - psnr: 33.0856 - ssim: 0.9665 - val_loss: 0.2207 - val_mean_squared_error: 0.0316 - val_psnr: 18.0754 - val_ssim: 0.7793
Epoch 4/30
1644/1644 [==============================] - 177s 108ms/step -[...]<jupyter_text>AE + Residual block (RB)  Loss function: MSE<jupyter_code>ae_rb_mse_model = AutoEncoder(
    encoder=ResnetEncoderv1(nb_list=[3,4,2], feature_maps=8),
    name="ae_rb_mse",
)(input_shape=workbench.image_shape)
ae_rb_mse_model.compile(loss='mean_squared_error', optimizer = 'adam', metrics=metrics)
ae_rb_mse_fitted = workbench.run(model=ae_rb_mse_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 220s 134ms/step - loss: 0.0067 - mean_squared_error: 0.0067 - psnr: 22.9834 - ssim: 0.4827 - val_loss: 0.0146 - val_mean_squared_error: 0.0146 - val_psnr: 18.7289 - val_ssim: 0.4913
Epoch 2/30
1644/1644 [==============================] - 217s 132ms/step - loss: 0.0027 - mean_squared_error: 0.0027 - psnr: 26.0709 - ssim: 0.7151 - val_loss: 0.0024 - val_mean_squared_error: 0.0024 - val_psnr: 26.6411 - val_ssim: 0.7827
Epoch 3/30
1644/1644 [==============================] - 218s 132ms/step - loss: 0.0021 - mean_squared_error: 0.0021 - psnr: 27.1833 - ssim: 0.8058 - val_loss: 0.0020 - val_mean_squared_error: 0.0020 - val_psnr: 27.4065 - val_ssim: 0.8335
Epoch 4/30
1644/1644 [==============================] - 217s 132ms/step - los[...]<jupyter_text>Loss function: SSIM<jupyter_code>ae_rb_ssim_model = AutoEncoder(
    encoder=ResnetEncoderv1(nb_list=[3,4,2], feature_maps=8),
    name="ae_rb_ssim",
)(input_shape=workbench.image_shape)
ae_rb_ssim_model.compile(loss=ssim_loss, optimizer = 'adam', metrics=metrics)
ae_rb_ssim_fitted = workbench.run(model=ae_rb_ssim_model, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 230s 140ms/step - loss: 0.2134 - mean_squared_error: 0.0450 - psnr: 19.4682 - ssim: 0.7866 - val_loss: 0.5717 - val_mean_squared_error: 0.3854 - val_psnr: 4.2281 - val_ssim: 0.4283
Epoch 2/30
1644/1644 [==============================] - 227s 138ms/step - loss: 0.1077 - mean_squared_error: 0.0043 - psnr: 25.4774 - ssim: 0.8923 - val_loss: 0.2202 - val_mean_squared_error: 0.0039 - val_psnr: 25.3879 - val_ssim: 0.7798
Epoch 3/30
1644/1644 [==============================] - 227s 138ms/step - loss: 0.0977 - mean_squared_error: 0.0042 - psnr: 25.8641 - ssim: 0.9024 - val_loss: 0.0984 - val_mean_squared_error: 0.0038 - val_psnr: 26.4119 - val_ssim: 0.9016
Epoch 4/30
1644/1644 [==============================] - 227s 138ms/step - loss[...]<jupyter_text>AE + sLC + RB (v1)  Loss function: MSE<jupyter_code>ae_slc_rb_mse_model_v1 = AutoEncoder(
    encoder=ResnetEncoderv1(nb_list=[3,4,2], feature_maps=8),
    name="ae_slc_rb_mse_v1",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_rb_mse_model_v1.compile(loss='mean_squared_error', optimizer = 'adam', metrics=metrics)
ae_slc_rb_mse_v1_fitted = workbench.run(model=ae_slc_rb_mse_model_v1, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 220s 134ms/step - loss: 0.0109 - mean_squared_error: 0.0109 - psnr: 23.0282 - ssim: 0.5615 - val_loss: 0.0034 - val_mean_squared_error: 0.0034 - val_psnr: 25.4447 - val_ssim: 0.6592
Epoch 2/30
1644/1644 [==============================] - 218s 133ms/step - loss: 0.0018 - mean_squared_error: 0.0018 - psnr: 28.2120 - ssim: 0.7437 - val_loss: 0.0082 - val_mean_squared_error: 0.0082 - val_psnr: 21.8322 - val_ssim: 0.7752
Epoch 3/30
1643/1644 [============================>.] - ETA: 0s - loss: 0.0016 - mean_squared_error: 0.0016 - psnr: 28.8968 - ssim: 0.8295
Epoch 00003: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.
1644/1644 [==============================] - 218s 132ms/step - loss: 0.0016 - mean_squared_erro[...]<jupyter_text>Loss function: SSIM<jupyter_code>ae_slc_rb_ssim_model_v1 = AutoEncoder(
    encoder=ResnetEncoderv1(nb_list=[3,4,2], feature_maps=8),
    name="ae_slc_rb_ssim_v1",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_rb_ssim_model_v1.compile(loss=ssim_loss, optimizer = 'adam', metrics=metrics)
ae_slc_rb_ssim_v1_fitted = workbench.run(model=ae_slc_rb_ssim_model_v1, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 232s 141ms/step - loss: 0.1201 - mean_squared_error: 0.0302 - psnr: 23.3479 - ssim: 0.8799 - val_loss: 0.0726 - val_mean_squared_error: 0.0035 - val_psnr: 27.0052 - val_ssim: 0.9274
Epoch 2/30
1644/1644 [==============================] - 229s 139ms/step - loss: 0.0585 - mean_squared_error: 0.0026 - psnr: 28.7042 - ssim: 0.9415 - val_loss: 0.1829 - val_mean_squared_error: 0.0026 - val_psnr: 28.4825 - val_ssim: 0.8171
Epoch 3/30
1644/1644 [==============================] - 229s 139ms/step - loss: 0.0522 - mean_squared_error: 0.0020 - psnr: 29.4470 - ssim: 0.9478 - val_loss: 0.0622 - val_mean_squared_error: 0.0019 - val_psnr: 29.2120 - val_ssim: 0.9378
Epoch 4/30
1644/1644 [==============================] - 229s 139ms/step - los[...]<jupyter_text>AE + sLC + RB (v2)  Loss function: MSE<jupyter_code>ae_slc_rb_mse_model_v2 = AutoEncoder(
    encoder=ResnetEncoderv2(nb_list=[3,4,2], feature_maps=8),
    name="ae_slc_rb_mse_v2",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_rb_mse_model_v2.compile(loss='mean_squared_error', optimizer = 'adam', metrics=metrics)
ae_slc_rb_mse_v2_fitted = workbench.run(model=ae_slc_rb_mse_model_v2, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 224s 136ms/step - loss: 0.0045 - mean_squared_error: 0.0045 - psnr: 27.0225 - ssim: 0.6635 - val_loss: 0.0013 - val_mean_squared_error: 0.0013 - val_psnr: 30.9129 - val_ssim: 0.7451
Epoch 2/30
1644/1644 [==============================] - 222s 135ms/step - loss: 0.0013 - mean_squared_error: 0.0012 - psnr: 30.8766 - ssim: 0.8117 - val_loss: 9.3088e-04 - val_mean_squared_error: 9.3136e-04 - val_psnr: 32.5089 - val_ssim: 0.8628
Epoch 3/30
1644/1644 [==============================] - 222s 135ms/step - loss: 9.3624e-04 - mean_squared_error: 9.3445e-04 - psnr: 32.3625 - ssim: 0.9047 - val_loss: 9.2374e-04 - val_mean_squared_error: 9.2449e-04 - val_psnr: 32.6850 - val_ssim: 0.8708
Epoch 4/30
1644/1644 [==============================][...]<jupyter_text>Loss function: SSIM<jupyter_code>ae_slc_rb_ssim_model_v2 = AutoEncoder(
    encoder=ResnetEncoderv2(nb_list=[3,4,2], feature_maps=8),
    name="ae_slc_rb_ssim_v2",
    skip_long_connections_function=layers.add
)(input_shape=workbench.image_shape)
ae_slc_rb_ssim_model_v2.compile(loss=ssim_loss, optimizer = 'adam', metrics=metrics)
ae_slc_rb_ssim_v2_fitted = workbench.run(model=ae_slc_rb_ssim_model_v2, epochs=30)<jupyter_output>WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
WARNING:tensorflow:sample_weight modes were coerced from
  ...
    to  
  ['...']
Train for 1644 steps, validate for 348 steps
Epoch 1/30
1644/1644 [==============================] - 235s 143ms/step - loss: 0.0812 - mean_squared_error: 0.0146 - psnr: 26.8620 - ssim: 0.9188 - val_loss: 0.3850 - val_mean_squared_error: 0.1728 - val_psnr: 7.9672 - val_ssim: 0.6150
Epoch 2/30
1644/1644 [==============================] - 232s 141ms/step - loss: 0.0369 - mean_squared_error: 0.0010 - psnr: 32.5258 - ssim: 0.9631 - val_loss: 0.3817 - val_mean_squared_error: 0.1586 - val_psnr: 8.2220 - val_ssim: 0.6183
Epoch 3/30
1644/1644 [==============================] - 232s 141ms/step - loss: 0.0323 - mean_squared_error: 8.9336e-04 - psnr: 33.2657 - ssim: 0.9676 - val_loss: 0.1098 - val_mean_squared_error: 0.0022 - val_psnr: 32.1704 - val_ssim: 0.8901
Epoch 4/30
1644/1644 [==============================] - 232s 141ms/step - l[...]
{'fraction_non_alphanumeric': 0.19347894903450458, 'fraction_numerical': 0.1347894903450459, 'mean_word_length': 4.663678737898889, 'pattern_counts': {'":': 0, '<': 49, '<?xml version=': 0, '>': 51, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 1, 'word_list_counts': {'cursed_substrings.json': 0, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '15', 'dir': 'jupyter-structured-clean-dedup', 'id': '98457', 'license': 'MIT', 'n_tokens_mistral': 7729, 'n_tokens_neox': 6008, 'n_words': 1190, 'path': 'notebooks/Architectures_depth4_fm8.ipynb', 'repo_name': 'epou/msc_thesis', 'size': '15795'}