<jupyter_start><jupyter_text>Exercise V: GANs Short summaryIn this exercise, we will design a generative network to generate the last rgb image given the first image. These folder has **three files**: - **configGAN.py:** this involves definitions of all parameters and data paths- **utilsGAN.py:** includes utility functions required to grab and visualize data - **runGAN.ipynb:** contains the script to design, train and test the network Make sure that before running this script, you created an environment and **installed all required libraries** such as keras. The dataThere exists also a subfolder called **data** which contains the traning, validation, and testing data each has both RGB input images together with the corresponding ground truth images. The exercisesAs for the previous lab all exercises are found below. The different 'Cells'This notebook contains several cells with python code, together with the markdown cells (like this one) with only text. Each of the cells with python code has a "header" markdown cell with information about the code. The table below provides a short overview of the code cells. |   |  CellName | CellType | Comment || :--- | :-------- | :-------- | :------- || 1 | Init | Needed | Sets up the environment|| 2 | Ex | Exercise 1| A class definition of a network model  || 3 | Loading | Needed | Loading parameters and initializing the model || 4 | Stats | Needed | Show data distribution | | 5 | Data | Needed | Generating the data batches || 6 | Debug | Needed | Debugging the data || 7 | Device | Needed | Selecting CPU/GPU || 8 | Init | Needed | Sets up the timer and other neccessary components || 9 | Training | Exercise 1-2 | Training the model   || 10 | Testing | Exercise 1-2| Testing the  method   |  In order for you to start with the exercise you need to run all cells. It is important that you do this in the correct order, starting from the top and continuing with the next cells. Later when you have started to work with the notebook it may be easier to use the command "Run All" found in the "Cell" dropdown menu. Writing the reportThere is no need to provide any report. However, implemented network architecuture and observed experimental results must be presented as a short presentation in the last lecture, May 28. 1) We first start with importing all required modules<jupyter_code>import os
from configGAN import *
cfg = flying_objects_config()
if cfg.GPU >=0:
    print("creating network model using gpu " + str(cfg.GPU))
    os.environ['CUDA_VISIBLE_DEVICES'] = str(cfg.GPU)
elif cfg.GPU >=-1:
    print("creating network model using cpu ")  
    os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"   # see issue #152
    os.environ["CUDA_VISIBLE_DEVICES"] = ""

import tensorflow as tf
from tensorflow import keras
from utilsGAN import *
from sklearn.metrics import confusion_matrix
# import seaborn as sns
from datetime import datetime
import imageio
from skimage import img_as_ubyte

import pprint
# import the necessary packages
from keras.models import Sequential
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import Conv3D, Conv2D, Conv1D, Convolution2D, Deconvolution2D, Cropping2D, UpSampling2D
from keras.layers import Input, Conv2DTranspose, ConvLSTM2D, TimeDistributed
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers import Concatenate, concatenate, Reshape
from keras.layers.core import Flatten
from keras.layers.core import Dropout
from keras.layers.core import Dense
from keras.preprocessing.image import ImageDataGenerator
from keras.optimizers import Adam, RMSprop
from keras.models import Model
from keras.callbacks import TensorBoard
from keras.applications.vgg16 import VGG16, preprocess_input, decode_predictions
from keras.layers import Input, merge
from keras.regularizers import l2
from keras.layers import Input, merge,AveragePooling2D, Convolution2D, MaxPooling2D, UpSampling2D, Reshape, core, Dropout, LeakyReLU
import keras.backend as kb
from tensorflow.keras.constraints import Constraint
from keras.initializers import RandomNormal
def wass_loss(y_true,y_pred):
    return kb.mean(y_true*y_pred)
# clip model weights to a given hypercube
class ClipConstraint(Constraint):
	# set clip value when initialized
	def __init__(self, clip_value):
		self.clip_value = clip_value
 
	# clip model weights to hypercube
	def __call__(self, weights):
		return kb.clip(weights, -self.clip_value, self.clip_value)
 
	# get the config
	def get_config(self):
		return {'clip_value': self.clip_value}<jupyter_output><empty_output><jupyter_text>2) Here, we have the network model class definition. In this class, the most important functions are **build_generator()** and **build_discriminator()**. As defined in the exercises section, your task is to update the both network architectures defined in these functions.<jupyter_code>class GANModel():
    def __init__(self, batch_size=32, inputShape=(64, 64, 3), dropout_prob=0.25): 
        self.batch_size = batch_size
        self.inputShape = inputShape
        self.dropout_prob = dropout_prob

        # Calculate the shape of patches
        patch = int(self.inputShape[0] / 2**4)
        self.disc_patch = (patch, patch, 1)
        opt = RMSprop(lr=0.00005)
        # Build and compile the discriminator
        self.discriminator = self.build_discriminator()
        #self.discriminator.compile(loss='mse', optimizer=Adam(0.0002, 0.5),metrics=['accuracy'])
        self.discriminator.compile(loss=wass_loss, optimizer=opt)
       
        # Build the generator
        self.generator = self.build_generator()

        # Input images and their conditioning images
        first_frame = Input(shape=self.inputShape)
        last_frame = Input(shape=self.inputShape)

        # By conditioning on the first frame generate a fake version of the last frame
        fake_last_frame = self.generator(first_frame)

        # For the combined model we will only train the generator
        self.discriminator.trainable = False
        
        # Discriminators determines validity of fake and condition first image pairs
        valid = self.discriminator([fake_last_frame, first_frame])
        #[1, 10]
        self.combined = Model(inputs=[last_frame, first_frame], outputs=[valid, fake_last_frame])
        self.combined.compile(loss={'Discriminator':wass_loss,'Generator':'mae'}, # mean squared and mean absolute errors
                              loss_weights={'Discriminator':1,'Generator':0},
                              optimizer=opt)

    def build_generator(self):
        inputs = Input(shape=self.inputShape)
        
        activation_fun = LeakyReLU(alpha=0.2)
        
        conv1= Conv2D(filters=32,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(inputs)
        conv1 = BatchNormalization(momentum=0.8)(conv1)
        conv1= Conv2D(filters=32,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv1)
        conv1 = BatchNormalization(momentum=0.8)(conv1)
        pool1 = Conv2D(filters=32,kernel_size=4,strides=2,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv1)#AveragePooling2D(pool_size=(2, 2))(conv1)
        
        conv2= Conv2D(filters=64,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(pool1)
        conv2 = BatchNormalization(momentum=0.8)(conv2)
        conv2= Conv2D(filters=64,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv2)
        conv2 = BatchNormalization(momentum=0.8)(conv2)
        pool2 = Conv2D(filters=64,kernel_size=4,strides=2,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv2)#AveragePooling2D(pool_size=(2, 2))(conv2)
        
        conv3= Conv2D(filters=128,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(pool2)
        conv3 = BatchNormalization(momentum=0.8)(conv3)
        conv3= Conv2D(filters=128,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv3)
        conv3 = BatchNormalization(momentum=0.8)(conv3)
        pool3 = Conv2D(filters=128,kernel_size=4,strides=2,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv3)#AveragePooling2D(pool_size=(2, 2))(conv3)
        
        conv4= Conv2D(filters=256,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(pool3)
        conv4 = BatchNormalization(momentum=0.8)(conv4)
        conv4= Conv2D(filters=256,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv4)
        conv4 = BatchNormalization(momentum=0.8)(conv4)
        drop4 = Dropout(0.5)(conv4)
        '''pool4 = MaxPooling2D(pool_size=(2, 2))(drop4)
        
        conv5= Conv2D(filters=256,kernel_size=3,activation=activation_fun,padding='same',kernel_initializer='he_normal')(pool4)
        conv5= Conv2D(filters=256,kernel_size=3,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv5)
        #pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)
        drop5 = Dropout(0.5)(conv5)
        
        
        up6 = UpSampling2D(size=(2, 2))(drop5)
        up6 = Conv2D(filters=256,kernel_size=3,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up6)
        up6 = Concatenate(axis=3)([drop4, up6])
        conv6 = Conv2D(filters=256,kernel_size=3,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up6)
        conv6 = Conv2D(filters=256,kernel_size=3,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv6)'''

        up7 = UpSampling2D(size=(2, 2))(conv4)
        up7 = Conv2D(filters=256,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up7)
        #up7 = BatchNormalization(momentum=0.8)(up7)
        up7 = Dropout(0.5)(up7)
        up7 = Concatenate(axis=3)([conv3, up7])
        conv7 = Conv2D(filters=256,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up7)
        conv7 = Dropout(0.5)(conv7)
        conv7 = Conv2D(filters=256,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv7)
        conv7 = Dropout(0.5)(conv7)

        up8 = UpSampling2D(size=(2, 2))(conv7)
        up8 = Conv2D(filters=128,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up8)
        up8 = Dropout(0.5)(up8)
        up8 = Concatenate(axis=3)([conv2, up8])
        conv8 = Conv2D(filters=128,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up8)
        conv8 = Dropout(0.5)(conv8)
        conv8 = Conv2D(filters=128,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv8)

        up9 = UpSampling2D(size=(2, 2))(conv8)
        up9 = Conv2D(filters=64,kernel_size=2,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up9)
        up9 = Dropout(0.5)(up9)
        up9 = Concatenate(axis=3)([conv1, up9])
        conv9 = Conv2D(filters=64,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(up9)
        conv9 = Dropout(0.5)(conv9)
        conv9 = Conv2D(filters=64,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv9)
        conv9 = Dropout(0.5)(conv9)
        conv9 = Conv2D(filters=32,kernel_size=4,activation=activation_fun,padding='same',kernel_initializer='he_normal')(conv9)
        conv9 = Dropout(0.5)(conv9)
        
        nbr_img_channels = self.inputShape[2]

        outputs = Conv2D(nbr_img_channels, (1, 1), activation='sigmoid')(conv9)

        model = Model(inputs=inputs, outputs=outputs, name='Generator')
        model.summary()
 
        '''inputs = Input(shape=self.inputShape)
        print(inputs.shape)
 
        down1 = Conv2D(32, (3, 3),padding='same')(inputs)
        down1 = Activation(activation_fun)(down1) 
        down1_pool = MaxPooling2D((2, 2), strides=(2, 2))(down1)
         
        down2 = Conv2D(64, (3, 3), padding='same')(down1_pool)
        down2 = Activation(activation_fun)(down2) 
         

        up1 = UpSampling2D((2, 2))(down2)
        up1 = concatenate([down1, up1], axis=3)
        up1 = Conv2D(256, (3, 3), padding='same')(up1) 
        up1 = Activation(activation_fun)(up1) 
        
        
        up2 = Conv2D(256, (3, 3), padding='same')(up1) 
        up2 = Activation(activation_fun)(up2) 
        
        nbr_img_channels = self.inputShape[2]
        outputs = Conv2D(nbr_img_channels, (1, 1), activation='sigmoid')(up2)

        model = Model(inputs=inputs, outputs=outputs, name='Generator')
        model.summary()'''
        
        return model

    def build_discriminator(self):
        const = ClipConstraint(0.01)
        last_img = Input(shape=self.inputShape)
        first_img = Input(shape=self.inputShape)
        activation_fun = LeakyReLU(alpha=0.2)
        # Concatenate image and conditioning image by channels to produce input
        combined_imgs = Concatenate(axis=-1)([last_img, first_img])
        init = RandomNormal(stddev=0.02)
        d1 = Conv2D(32, (4, 4), strides=2,kernel_initializer=init, padding='same',activation=activation_fun,kernel_constraint=const)(combined_imgs) 
        d1 = Dropout(0.5)(d1)
        d1 = Activation(activation_fun)(d1) 
        d2 = Conv2D(64, (4, 4), strides=2,kernel_initializer=init, padding='same',activation=activation_fun,kernel_constraint=const)(d1)
        d2 = Dropout(0.5)(d2)
        d2 = Activation(activation_fun)(d2) 
        d3 = Conv2D(128, (4, 4), strides=2,kernel_initializer=init, padding='same',activation=activation_fun,kernel_constraint=const)(d2)
        d3 = Dropout(0.5)(d3)
        d3 = Activation(activation_fun)(d3) 
        d4 = Conv2D(256, (4, 4), strides=2, kernel_initializer=init,padding='same',activation=activation_fun,kernel_constraint=const)(d3)
        d4 = Dropout(0.5)(d4)
        d4 = Activation(activation_fun)(d4) 
        validity = Dense(1)(d4)#

        model = Model([last_img, first_img], validity,name='Discriminator')
        model.summary()

        return model<jupyter_output><empty_output><jupyter_text>3) We import the network **hyperparameters** and build a simple network by calling the class introduced in the previous step. Please note that to change the hyperparameters, you just need to change the values in the file called **configPredictor.py.**<jupyter_code>image_shape = (cfg.IMAGE_HEIGHT, cfg.IMAGE_WIDTH, cfg.IMAGE_CHANNEL)
modelObj = GANModel(batch_size=cfg.BATCH_SIZE, inputShape=image_shape,
                                 dropout_prob=cfg.DROPOUT_PROB)<jupyter_output>Model: "Discriminator"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_98 (InputLayer)           [(None, 32, 32, 3)]  0                                            
__________________________________________________________________________________________________
input_99 (InputLayer)           [(None, 32, 32, 3)]  0                                            
__________________________________________________________________________________________________
concatenate_77 (Concatenate)    (None, 32, 32, 6)    0           input_98[0][0]                   
                                                                 input_99[0][0]                   
______________________________________________________________________________________[...]<jupyter_text>4) We call the utility function **show_statistics** to display the data distribution. This is just for debugging purpose.<jupyter_code>#### show how the data looks like
show_statistics(cfg.training_data_dir, fineGrained=False, title=" Training Data Statistics ")
show_statistics(cfg.validation_data_dir, fineGrained=False, title=" Validation Data Statistics ")
show_statistics(cfg.testing_data_dir, fineGrained=False, title=" Testing Data Statistics ")<jupyter_output>######################################################################
##################### Training Data Statistics #####################
######################################################################
total image number 	 10817
total class number 	 3
class square 	 3488 images
class triangle 	 3703 images
class circular 	 3626 images
######################################################################

######################################################################
##################### Validation Data Statistics #####################
######################################################################
total image number 	 2241
total class number 	 3
class circular 	 713 images
class square 	 783 images
class triangle 	 745 images
######################################################################

######################################################################
##################### Testing Data Statistics #####################
##########################[...]<jupyter_text>5) We now create batch generators to get small batches from the entire dataset. There is no need to change these functions as they already return **normalized inputs as batches**.<jupyter_code>nbr_train_data = get_dataset_size(cfg.training_data_dir)
nbr_valid_data = get_dataset_size(cfg.validation_data_dir)
nbr_test_data = get_dataset_size(cfg.testing_data_dir)
train_batch_generator = generate_lastframepredictor_batches(cfg.training_data_dir, image_shape, cfg.BATCH_SIZE)
valid_batch_generator = generate_lastframepredictor_batches(cfg.validation_data_dir, image_shape, cfg.BATCH_SIZE)
test_batch_generator = generate_lastframepredictor_batches(cfg.testing_data_dir, image_shape, cfg.BATCH_SIZE)
print("Data batch generators are created!")<jupyter_output>Data batch generators are created!<jupyter_text>6) We can visualize how the data looks like for debugging purpose<jupyter_code>if cfg.DEBUG_MODE:
    t_x, t_y,_ = next(train_batch_generator)
    print('train_x', t_x.shape, t_x.dtype, t_x.min(), t_x.max())
    print('train_y', t_y.shape, t_y.dtype, t_y.min(), t_y.max()) 
    #plot_sample_lastframepredictor_data_with_groundtruth(t_x, t_y, t_y)
    pprint.pprint (cfg)<jupyter_output>train_x (30, 32, 32, 3) float32 0.0 1.0
train_y (30, 32, 32, 3) float32 0.0 1.0
{'BATCH_SIZE': 30,
 'CLASSES': ['square_red',
             'square_green',
             'square_blue',
             'square_yellow',
             'triangle_red',
             'triangle_green',
             'triangle_blue',
             'triangle_yellow',
             'circular_red',
             'circular_green',
             'circular_blue',
             'circular_yellow'],
 'DATA_AUGMENTATION': True,
 'DEBUG_MODE': True,
 'DROPOUT_PROB': 0.5,
 'GPU': 0,
 'IMAGE_CHANNEL': 3,
 'IMAGE_HEIGHT': 32,
 'IMAGE_WIDTH': 32,
 'LEARNING_RATE': 0.01,
 'LR_DECAY_FACTOR': 0.1,
 'NUM_EPOCHS': 5,
 'PRINT_EVERY': 20,
 'SAVE_EVERY': 1,
 'SEQUENCE_LENGTH': 10,
 'testing_data_dir': '../data/FlyingObjectDataset_10K/testing',
 'training_data_dir': '../data/FlyingObjectDataset_10K/training',
 'validation_data_dir': '../data/FlyingObjectDataset_10K/validation'}<jupyter_text>7) Start timer and init matrices<jupyter_code>start_time = datetime.now()
# Adversarial loss ground truths
valid = -np.ones((cfg.BATCH_SIZE,) + modelObj.disc_patch)
fake = np.ones((cfg.BATCH_SIZE,) + modelObj.disc_patch)#np.zeros((cfg.BATCH_SIZE,) + modelObj.disc_patch)
# log file
output_log_dir = "./logs/{}".format(datetime.now().strftime("%Y%m%d-%H%M%S"))
if not os.path.exists(output_log_dir):
    os.makedirs(output_log_dir)<jupyter_output><empty_output><jupyter_text>8) We can now feed the training and validation data to the network. This will train the network for **some epochs**. Note that the epoch number is also predefined in the file called **configGAN.py.**<jupyter_code>from tqdm.notebook import tqdm, trange

import imageio 
test_first_imgs, test_last_imgs,test_labels = next(test_batch_generator)

plot_first_imgs, plot_last_imgs, plot_lables_imgs = next(valid_batch_generator)

# plot_lables_imgs = label_encoder(plot_lables_imgs)#mapLabels(plot_lables_imgs)
# print(plot_lables_imgs.shape)

disc_losses_val = []
disc_real_losses = []
disc_fake_losses=[]
generator_losses_val = []
disc_accuracies_val_fake = []
disc_accuracies_val_real = []
conditional = True
plot_epoch = False
plots = []

                

fake_last_frames_plot = modelObj.generator.predict(plot_first_imgs)
# Train the discriminator with combined loss  
d_loss_real = modelObj.discriminator.evaluate([plot_last_imgs, plot_first_imgs], valid)#modelObj.discriminator.train_on_batch([plot_last_imgs, plot_first_imgs,plot_lables_imgs], valid)
d_loss_fake = modelObj.discriminator.evaluate([fake_last_frames_plot, plot_first_imgs], fake)#modelObj.discriminator.train_on_batch([fake_last_frames_plot, plot_first_imgs,plot_lables_imgs], fake)
d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

# Train the generator
g_loss = modelObj.combined.evaluate([plot_last_imgs, plot_first_imgs], [valid, plot_last_imgs])

disc_real_losses.append(d_loss_real)
disc_fake_losses.append(d_loss_fake)
disc_losses_val.append(d_loss)
generator_losses_val.append(g_loss[0])

left, width = .25, .5
bottom, height = .25, .5
right = left + width
top = bottom + height
rows = 2
fig, axs = plt.subplots(nrows=rows, ncols=4, figsize=(10, 8))
for ax, batch_i in zip(axs, range(rows)):

    ax[0].text(0.5*(left+right), 0.5*(bottom+top), 0,
    horizontalalignment='center',
    verticalalignment='center',
    fontsize=15)
    ax[0].axis('off')
    for i, img in zip(range(1,4), [["First Img", plot_first_imgs[batch_i]], ["Last Img", plot_last_imgs[batch_i]], ["Fake Last Img", fake_last_frames_plot[batch_i]]]):
        ax[i].imshow(img[1])
        if(batch_i == 0):
            ax[i].set_title(img[0])
        ax[i].axis('auto')

plt.tight_layout()
plt.show()
fig.canvas.draw()       # draw the canvas, cache the renderer
image = np.frombuffer(fig.canvas.tostring_rgb(), dtype='uint8')
image  = image.reshape(fig.canvas.get_width_height()[::-1] + (3,))
plots.append(image)


for epoch in range(20):#cfg.NUM_EPOCHS):
    print("=====RUN {} of {}=====".format(epoch+1,10))
    steps_per_epoch = (nbr_train_data // 30) 
    for batch_i in trange(steps_per_epoch):
        first_frames, last_frames,_ = next(train_batch_generator)
        if first_frames.shape[0] == cfg.BATCH_SIZE: 
            #print(first_frames.shape)
            
            #=========================TRAIN NETWORK ==================================
            
            for x in range(5):
                fake_last_frames = modelObj.generator(first_frames)

                # Train the discriminator with combined loss  
                d_loss_real = modelObj.discriminator.train_on_batch([last_frames, first_frames], valid)
                d_loss_fake = modelObj.discriminator.train_on_batch([fake_last_frames, first_frames], fake)
                d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)


            g_loss = modelObj.combined.train_on_batch( [last_frames,first_frames], [valid,last_frames])


            elapsed_time = datetime.now() - start_time 

            if  batch_i in [1,90,180,270,358]:
                print ("[Epoch %d/%d] [Batch %d/%d] [D loss: %f] [G loss: %f] time: %s" % (epoch, cfg.NUM_EPOCHS,
                                                                                               batch_i,
                                                                                               steps_per_epoch,
                                                                                               d_loss, 
                                                                                               g_loss[0],
                                                                                               elapsed_time))
                
            #==========================================================================
    #CGAN
    fake_last_frames = modelObj.generator.predict(plot_first_imgs)

    # Train the discriminator with combined loss  
    print("====Real=====")
    d_loss_real = modelObj.discriminator.evaluate([plot_last_imgs, plot_first_imgs], valid)
    print("====Fake=====")
    d_loss_fake = modelObj.discriminator.evaluate([fake_last_frames_plot, plot_first_imgs], fake)
    d_loss = 0.5 * np.add(d_loss_real, d_loss_fake)

    # Train the generator
    g_loss = modelObj.combined.evaluate([plot_last_imgs, plot_first_imgs], [valid, plot_last_imgs],verbose=0)

#     disc_accuracies_val_fake.append(d_loss_fake[1])
#     disc_accuracies_val_real.append(d_loss_real[1])
    disc_real_losses.append(d_loss_real)
    disc_fake_losses.append(d_loss_fake)
    disc_losses_val.append(d_loss)
    generator_losses_val.append(g_loss[0])

    left, width = .25, .5
    bottom, height = .25, .5
    right = left + width
    top = bottom + height
    #fake_last_plotFrames = modelObj.generator.predict([plot_first_imgs,plot_lables_imgs])
    rows = 2
    fig, axs = plt.subplots(nrows=rows, ncols=4, figsize=(10, 8))
    for ax, batch_i in zip(axs, range(rows)):

        ax[0].text(0.5*(left+right), 0.5*(bottom+top), epoch + 1,
        horizontalalignment='center',
        verticalalignment='center',
        fontsize=15)
        ax[0].axis('off')
        for i, img in zip(range(1,4), [["First Img", plot_first_imgs[batch_i]], ["Last Img", plot_last_imgs[batch_i]], ["Fake Last Img", fake_last_frames[batch_i]]]):
            ax[i].imshow(img[1])
            if(batch_i == 0):
                ax[i].set_title(img[0])
            ax[i].axis('auto')


    plt.tight_layout()
    plt.show()
    fig.canvas.draw()       # draw the canvas, cache the renderer
    image = np.frombuffer(fig.canvas.tostring_rgb(), dtype='uint8')
    image  = image.reshape(fig.canvas.get_width_height()[::-1] + (3,))
    plots.append(image)

    print("=======VALIDATION=======")
    print("D loss: {}  G loss {}".format(d_loss,g_loss[0]))
    #print("D acc Real: {} D acc Fake: {}".format(d_loss_real[1],d_loss_fake[1]))

    
imageio.mimsave('./powers.gif', plots, fps=3)
fig = plt.figure(figsize=(10,10))

plt.plot(disc_losses_val,label="Discriminator Loss",color="b",linestyle="--",marker='o')
plt.title('Discriminator loss')
plt.show()

fig = plt.figure(figsize=(10,10))
plt.plot(disc_accuracies_val_real,label="Real",color="b",linestyle="--",marker='o')
plt.plot(disc_accuracies_val_fake,label="Fake",color="r",linestyle="--",marker='o')
plt.legend()
plt.title('Discriminator Acc')
plt.show()


fig = plt.figure(figsize=(10,10))
plt.plot(generator_losses_val,label="Generator Loss",color="r",linestyle="--",marker='o')
plt.title('Generator')
plt.show()
rows = 5
fig, axs = plt.subplots(nrows=rows, ncols=4, figsize=(8, 2*rows))

# build a rectangle in axes coords
left, width = .25, .5
bottom, height = .25, .5
right = left + width
top = bottom + height


for ax, batch_i in zip(axs, range(rows)):
    test_first_imgs, test_last_imgs,test_lables = next(test_batch_generator)
    #test_lables = label_encoder(test_lables)

    test_fake_last_imgs = modelObj.generator.predict([test_first_imgs]) 
    #test_img_name = output_log_dir + "/gen_img_test_" + str(batch_i) + ".png"
    #merged_img = np.vstack((test_first_imgs[0],test_last_imgs[0],test_fake_last_imgs[0]))
    #imageio.imwrite(test_img_name, img_as_ubyte(merged_img))
    ax[0].text(0.5*(left+right), 0.5*(bottom+top), batch_i,
    horizontalalignment='center',
    verticalalignment='center',
    fontsize=15)
    ax[0].axis('off')
    for i, img in zip(range(1,4), [["First Img", test_first_imgs[0]], ["Last Img", test_last_imgs[0]], ["Fake Last Img", test_fake_last_imgs[0]]]):
        ax[i].imshow(img[1])
        if(batch_i == 0):
            ax[i].set_title(img[0])
        ax[i].axis('auto')


plt.tight_layout()
plt.show()<jupyter_output><empty_output><jupyter_text>9) We can test the model with 100 test data which will be saved as images<jupyter_code>for batch_i in range(100):
    test_first_imgs, test_last_imgs = next(test_batch_generator)
    test_fake_last_imgs = modelObj.generator.predict(test_first_imgs) 

    test_img_name = output_log_dir + "/gen_img_test_" + str(batch_i) + ".png"
    merged_img = np.vstack((test_first_imgs[0],test_last_imgs[0],test_fake_last_imgs[0]))
    imageio.imwrite(test_img_name, img_as_ubyte(merged_img))<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.14434898016205644, 'fraction_numerical': 0.02752165409332216, 'mean_word_length': 4.074074074074074, 'pattern_counts': {'":': 0, '<': 33, '<?xml version=': 0, '>': 35, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 27, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '9', 'dir': 'jupyter-structured-clean-dedup', 'id': '68089', 'license': 'MIT', 'n_tokens_mistral': 9519, 'n_tokens_neox': 8880, 'n_words': 1832, 'path': 'lab_gan/runGAN-1-WAS.ipynb', 'repo_name': 'AndreasH96/ArtificialFlyingObjects', 'size': '28632'}