Disentangling underlying generative factors of a data distribution is important for interpretability and generalizable representations. In this paper,  we introduce two novel disentangling methods. Our first method, Unlabeled Disentangling GAN (UD-GAN, unsupervised), decomposes the latent noise by generating similar/dissimilar image pairs and it learns a distance metric on these pairs with siamese networks and a contrastive loss. This pairwise approach provides consistent representations for similar data points. Our second method (UD-GAN-G, weakly supervised) modifies the UD-GAN with user-defined guidance functions, which restrict the information that goes into the siamese networks. This constraint helps UD-GAN-G to focus on the desired semantic variations in the data. We  show  that  both  our  methods  outperform  existing  unsupervised approaches in quantitative metrics that measure semantic accuracy of the learned representations. In addition, we illustrate that simple guidance functions we use in UD-GAN-G allow us to directly capture the desired variations in the data. Generative Adversarial Networks (GANs) are generative model estimators, where two neural networks (generator and discriminator) are trained in an adversarial setting, so that likelihood-based probabilistic modeling is not necessary. This works particularly well for sampling from a complex probability distribution, such as images. Although GANs yield realistic looking images BID18 , the original formulation in only allows for randomly sampling from the data distribution without disentangled structural or semantic control over the generated data points.One way to disentangle the generation process is to use conditional GANs BID16 BID17 . These models modify the generator by conditioning it with supervised labels. Then, they either take the same labels as input in the discriminator BID16 and measure the image-label compatibility, or classify the correct label at the output, given the generated image BID17 . Conditional GANs rely on a dataset with labels, which might not always be available or might be time-consuming to collect.In this paper, we propose two GAN-based methods that learns disentangled representations without using labeled data. Our first method, Unlabeled Disentangling GAN (UD-GAN), generates image pairs, then embeds them with Siamese Networks BID2 , and finally learns a distance metric on a disentangled representation space. Whereas our second method, UD-GAN-G, uses guidance functions to restrict the input to our siamese networks, so that they capture desired semantic variations. In this paper we introduced UD-GAN and UD-GAN-G, novel GAN formulations which employ Siamese networks with contrastive losses in order to make slices of the latent noise space disentangled and more semantically meaningful. Our experiments encompassed guided and unguided approaches for the embedding networks, and illustrated how our methods can be used for semantically meaningful image manipulation. Our qualitative and quantiative results confirm that our method can adjust well to the intrinsic factors of variation of the data and outperform the current state-of-the-art methods on the CelebA and 2D Shapes datasets. In future work, we plan to investigate more powerful forms of embedders, e.g. extracting information from pre-trained networks for semantic segmentation and landmark detection. This allows for even more powerful novel image manipulation techniques.
We use Siamese Networks to guide and disentangle the generation process in GANs without labeled data.
{"src": "scitldr-AIC_train", "paper_id": "H1e0-30qKm"}