The text-to-image synthesis involves the subtask image-text matching, which refers to learning the joint image-text representation to maximize the semantic similarity for an image-sentence pair.
Tao et al. [1]} propose the Deep Attentional Multimodal Similarity Model (DAMSM) to learn the fine-grained image-text representation for the text-to-image synthesis. The DAMSM (Figure REF a) trains an image encoder and a text encoder jointly to encode sub-regions of the image and words of the sentence to a common semantic space, and computes a fine-grained image-text matching loss for image generation. However, the variations exist in the text representations corresponding to the same image, which leads to the generated images deviating from the ground truth image. To address this issue, we utilize the contrastive learning approach to push together the text representations related to the identical image and push away the text representations related to different images.
