Recently, a great number of studies [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, [13]}, [14]}, [15]}, [16]}, [17]}, [18]}, [19]}, [20]} present
promising results on the text-to-image synthesis task, most of which make use of GANs as the backbone model.
We briefly summarizes some of them that are most related to our approach.
Zhang et al. [2]} propose the stacked GAN architecture which produces images from low-resolution to high-resolution. AttnGAN [4]} presents an attention mechanism, where the Deep Attentional Multimodal Similarity Model (DAMSM) is able to compute the similarity between the generated image and the caption using both the global sentence level information and the fine-grained word level information. DM-GAN [8]} introduces a dynamic memory generative adversarial network to generate high-quality images. It utilizes a dynamic memory module to refine the initial generated image, a memory writing gate to highlight the relevant text information and a response gate to update image representations. SD-GAN [9]} employs a Siamese structure with a pair of texts as input and trains the model with the contrastive loss. The conditional batch normalization is adopted for fine-grained image generation. Compared with the Siamese structure in SD-GAN, our approach is derived from the recent development of the contrastive learning paradigm, therefore, it has the advantage of better performance and less computational cost. Furthermore, we generalize our approach so that it can be applied to the existing GAN-based models for text-to-image synthesis.
XMC-GAN [20]} has also applied the contrastive learning approach in the text-to-image generation. However, the objectives of contrastive loss in our approach are different from them in XMC-GAN. We compute the contrastive losses of the caption-caption pair and the fake-fake
pair, which are complementary to the contrastive losses in XMC-GAN. In this work, we choose AttnGAN and DM-GAN as the base models to evaluate our approach.
