This paper proposes a dual variational autoencoder (DualVAE), a framework for generating images corresponding to multiclass labels. Recent research on conditional generative models, such as the Conditional VAE, exhibit image transfer by changing labels. However, when the dimension of multiclass labels is large, these models cannot change images corresponding to labels, because learning multiple distributions of the corresponding class is necessary to transfer an image. This leads to the lack of training data. Therefore, instead of conditioning with labels, we condition with latent vectors that include label information. DualVAE divides one distribution of the latent space by linear decision boundaries using labels. Consequently, DualVAE can easily transfer an image by moving a latent vector toward a decision boundary and is robust to the missing values of multiclass labels. To evaluate our proposed method, we introduce a conditional inception score (CIS) for measuring how much an image changes to the target class. We evaluate the images transferred by DualVAE using the CIS in CelebA datasets and demonstrate state-of-the-art performance in a multiclass setting. Recent conditional generative models have shown remarkable success in generating and transferring images. Specifically, a conditional variational autoencoder (CVAE) BID4 can generate conditional images by learning the latent space Z that corresponds to multiclass labels. In addition, StarGAN BID1 and FaderNetworks BID5 can generate images corresponding to multiple domains by conditioning with domains such as attributes.However, when the dimension of the multiclass is increased, these models cannot transfer the images corresponding to one arbitrary domain (an element of a multiclass label). The possible reasons are the following. For simplicity, we consider a binary multiclass classification. To transfer an image of a certain class, it is necessary to learn the distributions of the corresponding class. That is, assuming that the number of classes in the multiclass is N, conditional models need to create 2 N distributions. However, when N is large, training is difficult as O(2 N ) training samples will be required.Hence, instead of conditioning with labels, we propose DualVAE, which conditions with latent vectors that include label information. DualVAE divides one distribution of the latent space by N linear decision boundaries which need to learn only O(N ) parameters by adding another decoder p w (y|z) to a variational autoencoder (VAE) BID3 . DualVAE assumes that a label is a linear combination of vectors of the latent space and the dual latent space. There are two advantages to the DualVAE decoder p w (y|z) being a linear model. First, DualVAE can easily transfer an image by moving a latent vector toward a decision boundary. Next, DualVAE is robust to the missing values of multiclass labels.In addition to this method, we propose the conditional inception score (CIS), a new metric for conditional transferred images. Although the evaluation methods often used in the generation models are the Inception Score (IS) BID9 and the Fréchet Inception Distance BID2 , they are used for evaluating the diversity of images and not suitable for evaluating transferred images conditioned with domains such as attributes or classes. Therefore, we propose a new metric to evaluate two properties: the first property pertains to whether images in one domain are transferred properly to images in another domain; the second property pertains to whether images in one domain Figure 1 : Conditional VAE learns 2 n distributions for each binary multiclass label when the number of class is n. DualVAE learns n decision boundaries for dividing a distribution of latent space. u 1 is a parameter of a decision boundary, which we call a dual vector.transferred to images in another domain can preserve the original properties. By using the CIS, we compare DualVAE with other methods that can perform image-to-image translations for multiple domains.In summary, the contributions from this study are as follows: 1) We introduce DualVAE, a method for transferring images corresponding to multiclass labels and demonstrate that images can be transferred quantitatively and qualitatively. 2) We propose the CIS, a new metric that can evaluate transferred images corresponding to multiclass labels. We proposed DualVAE, a simple framework for generating and transferring images corresponding to multiclass labels. Further, we introduced the CIS, a new metric for measuring how much of an image corresponding to the change of labels could be generated. The decoder of DualVAE was a simple linear model in this study; however, we would like to test more complex models in the future.
 a new framework using dual space for generating images corresponding to multiclass labels when the number of class is large
{"src": "scitldr-AIC_train", "paper_id": "S1lnmLLKu4"}