<jupyter_start><jupyter_text>Conditional GAN (CGAN) In conditional GANs, we get to condition the output of the generator and thus the characteristics of the generated images. In order to do that, we use some labels, which indicate the desired characteristics. For the GAN to learn the meaning of each label, we train it with images that satisfy the specific conditions every time the corresponding label is present. For example if we use the mnist dataset and we want to condition the generator on the number of the output digit, the label can be an integer in [0, 9]. Every time we train the models, we pass this label as input both for the real and the generated images. Based on the real images, the discriminator learns to corelate the labels with the characteristics of the digits and the generator learns to produce images with these characteristics when given a label. Here, we will see a simple example of how we can "pass" this information to the models.The template of the code is based on [this](https://colab.research.google.com/drive/1jdSI0zGTwrZzQQ0QIJ1pq8oGsPSb_DFD) template of the simple GAN.  Imports<jupyter_code>from keras.models import Model
from keras.layers import Input, Dense, BatchNormalization, Reshape, Flatten
from keras.layers import Embedding, multiply, add, concatenate #NEW!
from keras.layers.advanced_activations import LeakyReLU
from keras.datasets import mnist, fashion_mnist
from keras.optimizers import Adam

import numpy as np
import matplotlib.pyplot as plt
from IPython.display import Image<jupyter_output><empty_output><jupyter_text>Function to build the generator This function constructs the generator of the CGAN. The difference from the simple GAN is that now we also input the label. The label is one integer (tensor of shape (1,) and type int32). In order to use this information, we have to somehow "merge" it with the "information" of the noise. The first step is to transform this integer to an embedding. The embeding is a fixed size vector that encloses the information of the given input. In keras this is done using the [Embedding](https://keras.io/layers/embeddings/embedding) layer. We can define the output size of the embedding to be any positive number, however we will choose to set it to be equal with the noise size, so that it can be compatible with different kinds of merging operations.The next step is to merge this embedded label with the input noise. For this we can use any of the [merge](https://keras.io/layers/merge/) layers of keras. Here we show the case of concatenate, how ever one can replace it with another one (e.g. add or multiply). The merging layer will take the two tensors (noise and embedded_label) and will output a single merged tensor. this will be our new model input. The rest of the model is the same as in the simple GAN. This time the generator will take two inputs, so when we define it we use a list of input tensors instead of a single input tensor (`Model([noise, label], img)`).<jupyter_code>def build_generator(noise_size, img_shape, classes_num):
  
  noise = Input((noise_size,))
  label = Input((1,), dtype='int32') #NEW!
  
  embedded_label = Flatten()(Embedding(classes_num, noise_size)(label)) #NEW!
  model_input = concatenate([noise, embedded_label]) #NEW!
  
  x = Dense(256)(model_input) #CHANGED!
  x = BatchNormalization()(x)
  x = LeakyReLU(alpha=0.2)(x)
  
  x = Dense(512)(x)
  x = BatchNormalization()(x)
  x = LeakyReLU(alpha=0.2)(x)
  
  x = Dense(1024)(x)
  x = BatchNormalization()(x)
  x = LeakyReLU(alpha=0.2)(x)
  
  x = Dense(np.prod(img_shape), activation='tanh')(x)
  img = Reshape(img_shape)(x)
    
  generator = Model([noise, label], img) #CHANGED!
  return generator<jupyter_output><empty_output><jupyter_text>Function to build the discriminator This function constructs the discriminator of the CGAN. As it was at the case of the generator, we have to somehow merge the information of the label with the model's input, in this case the image. In order to do this, again we take the input label which is an integer and transform it to an embedding. Then, we merge this embedded label with the image. For the merging we can, once again, use any of the available merge layers of keras. Here we show the case of concatenation. Since the first layer of the discriminator will be a Dense layer, we Flatten both the image and the embedding before the concatenation. The rest of the model is the same as in the simple GAN. This time the discriminator will take two inputs, so when we define it we use a list of input tensors instead of a single input tensor (`Model([img, label], validity)`).<jupyter_code>def build_discriminator(img_shape, classes_num):
    
  img = Input(img_shape)
  label = Input((1,), dtype='int32') #NEW!
  
  f_img = Flatten()(img)
  embedded_label = Flatten()(Embedding(classes_num, np.prod(img_shape))(label)) #NEW!
  model_input = concatenate([f_img, embedded_label]) #NEW!
  
  x = Dense(1024)(model_input)
  x = LeakyReLU(alpha=0.2)(x)
  
  x = Dense(512)(x)
  x = LeakyReLU(alpha=0.2)(x)
  
  x = Dense(256)(x)
  x = LeakyReLU(alpha=0.2)(x)
  
  validity = Dense(1, activation='sigmoid')(x)
  
  discriminator = Model([img, label], validity) #CHANGED!
  return discriminator<jupyter_output><empty_output><jupyter_text>Function to compile the models<jupyter_code>def get_compiled_models(generator, discriminator, noise_size):
  
  optimizer = Adam(0.0002, 0.5)
  
  discriminator.compile(optimizer, loss='binary_crossentropy', metrics=['accuracy'])
  discriminator.trainable = False
  
  noise = Input((noise_size,))
  label = Input((1,), dtype='int32') #NEW!
  img = generator([noise, label]) #CHANGED!
  validity = discriminator([img, label]) #CHANGED!
  combined = Model([noise, label], validity) #CHANGED!
  
  combined.compile(optimizer, loss='binary_crossentropy')
  
  return generator, discriminator, combined<jupyter_output><empty_output><jupyter_text>Function to sample and save generated images<jupyter_code>def sample_imgs(generator, noise_size, step, plot_img=True):
  
  r, c = 10, 5 #CHANGED!
  imgs = []
  for i in range(c):
    noise = np.random.normal(0, 1, (r, noise_size))
    sampled_labels = np.arange(r).reshape(-1, 1) #NEW!
    img = generator.predict([noise, sampled_labels]) #CHANGED!
    img = img / 2 + 0.5
    imgs.append(img)
  
  figsize = 1 * c, 1 * r
  fig, axs = plt.subplots(r, c, figsize=figsize)
  
  for i in range(r):
    for j in range(c):
      axs[i, j].imshow(imgs[j][i], cmap='gray')
      axs[i, j].axis('off')
  plt.subplots_adjust(wspace=0.1, hspace=0.1)
  fig.savefig(f'/content/images/{step}.png')
  if plot_img:
    plt.show()
  plt.close()<jupyter_output><empty_output><jupyter_text>Function to train the models For the training of the CGAN, we have to give to both the discriminator and the combined model the labels of the digits. The labels are integers taken by the mnist dataset. For each selected real image, we also take the corresponding label. We use these pairs of image-label as input for all the models. This way the models learn to recognize (discriminator) and produce (generator) images that are related to the labels. The rest of the function is the same as in the simple GAN.<jupyter_code>def train(models, noise_size, img_shape, batch_size, steps):
  
  generator, discriminator, combined = models
  #get real data
  (X_train, Y_train), (X_val, Y_val) = fashion_mnist.load_data() #CHANGED!
  mnist_imgs = np.concatenate((X_train, X_val)) / 127.5 - 1 
  mnist_labels = np.concatenate((Y_train, Y_val)) #CHANGED!
  
  for step in range(1, steps + 1):
    # train discriminator
    inds = np.random.randint(0, mnist_imgs.shape[0], batch_size)
    real_imgs, labels = mnist_imgs[inds], mnist_labels[inds] #CHANGED!
    real_validity = np.ones(batch_size)
    
    noise = np.random.normal(0, 1, (batch_size, noise_size))
    gen_imgs = generator.predict([noise, labels]) #CHANGED!
    gen_validity = np.zeros(batch_size)
    
    r_loss = discriminator.train_on_batch([real_imgs, labels], real_validity) #CHANGED!
    g_loss = discriminator.train_on_batch([gen_imgs, labels], gen_validity) #CHANGED!
    disc_loss = np.add(r_loss, g_loss) / 2
    
    # train generator
    noise = np.random.normal(0, 1, (batch_size, noise_size))
    gen_validity = np.ones(batch_size)
    gen_loss = combined.train_on_batch([noise, labels], gen_validity) #CHANGED!
    
    #print progress
    if step % 100 == 0:
      print('step: %d, D_loss: %f, D_accuracy: %.2f%%, G_loss: %f' % (step, disc_loss[0],
                                                                      disc_loss[1] * 100, gen_loss))
    
    # save_samples
    if step % 100 == 0:
      sample_imgs(generator, noise_size, step)<jupyter_output><empty_output><jupyter_text>Define hyperparameters<jupyter_code>noise_size = 100
img_shape = 28, 28
batch_size = 32
steps = 5000
classes_num = 10 #NEW!<jupyter_output><empty_output><jupyter_text>Generate the models<jupyter_code>generator = build_generator(noise_size, img_shape, classes_num) #CHANGED!
discriminator = build_discriminator(img_shape, classes_num) #CHANGED!
compiled_models = get_compiled_models(generator, discriminator, noise_size)<jupyter_output><empty_output><jupyter_text>Train the models<jupyter_code>%rm -r /content/images
%mkdir /content/images
train(compiled_models, noise_size, img_shape, batch_size, steps)<jupyter_output><empty_output><jupyter_text>Display samples<jupyter_code>Image('/content/images/%d.png' % 5000)<jupyter_output><empty_output>
{'fraction_non_alphanumeric': 0.0952581664910432, 'fraction_numerical': 0.01285563751317176, 'mean_word_length': 4.60271546635183, 'pattern_counts': {'":': 0, '<': 41, '<?xml version=': 0, '>': 41, 'https://': 3, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 3, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
{'chain_length': '10', 'dir': 'jupyter-structured-clean-dedup', 'id': '219764', 'license': 'MIT', 'n_tokens_mistral': 3145, 'n_tokens_neox': 3001, 'n_words': 1059, 'path': 'Conditional_GAN/Simple_CGAN.ipynb', 'repo_name': 'gnuevo/Intro-to-GANs', 'size': '9490'}