This paper presents the Variation Network (VarNet), a  generative model providing means to manipulate the high-level attributes of a given input. The originality of our approach is that VarNet is not only capable of handling pre-defined attributes but can also learn the relevant attributes of the dataset by itself.   These two settings can be easily combined  which makes VarNet applicable for a wide variety of tasks. Further, VarNet has a sound probabilistic interpretation which grants us with  a novel way to navigate in the latent spaces as well as means to control how the  attributes are learned. We demonstrate  experimentally that this model is capable of performing interesting input manipulation  and that the learned attributes are relevant and interpretable. We focus on the problem of generating variations of a given input in an intended way. This means that given some input element x, which can be considered as a template, we want to generate transformed versions of x with different high-level attributes. Such a mechanism is of great use in many domains such as image edition since it allows to edit images on a more abstract level and is of crucial importance for creative uses since it allows to generate new content.More precisely, given a dataset D = {(x (1) , m (1) ), . . . , (x (N ) , m (N ) )} of N labeled elements (x, m) ∈ X × M, where X stands for the input space and M for the metadata space, we would like to obtain a model capable of learning a relevant attribute space Ψ ⊂ R d for some integer d > 0 and meaningful attribute functions φ : X × M → Ψ that we can then use to control generation.In a great majority of the recent proposed methods BID13 ; BID16 , these attributes are assumed to be given. We identify two shortcomings: labeled data is not always available and this approach de facto excludes attributes that can be hard to formulate in an absolute way. The novelty of our approach is that these attributes can be either learned by the model (we name them free attributes) or imposed (fixed attributes). This problem is an ill-posed one on many aspects. Firstly, in the case of fixed attribute functions φ, there is no ground truth for variations since there is no x with two different attributes. Secondly, it can be hard to determine if a learned free attribute is relevant. However, we provide empirical evidence that our general approach is capable of learning such relevant attributes and that they can be used for generating meaningful variations.In this paper, we introduce the Variation Network (VarNet), a probabilistic neural network which provides means to manipulate an input by changing its high-level attributes. Our model has a sound probabilistic interpretation which makes the variations obtained by changing the attributes statistically meaningful. As a consequence, this probabilistic framework provides us with a novel mechanism to "control" or "shape" the learned free attributes which then gives interpretable controls over the variations. This architecture is general and provides a wide range of choices for the design of the attribute function φ: we can combine both free and fixed attributes and the fixed attributes can be either continuous or discrete.Our contributions are the following:• A widely applicable encoder-decoder architecture which generalizes existing approaches BID11 ; BID14 ; BID13 The input x,x are in X , the input space and the metadata m is in M, the metadata space. The latent template code z * lies in Z * , the template space, while the latent variable z lies in Z the latent space. The variable u is sampled from a zero-mean unitvariance normal distribution. Finally, the features φ(x, m) are in Ψ, the attribute space. The Neural Autoregressive Flows (NAF) BID10 are represented using two arrows, one pointing to the center of the other one; this denotes the fact that the actual parameters of first neural network are obtained by feeding meta-parameters into a second neural network. The discriminator D acts on Z * × Ψ.• An easy-to-use framework: any encoder-decoder architecture can be easily transformed into a VarNet in order to provide it with controlled input manipulation capabilities,• A novel and statistically sound approach to navigate in the latent space,• Ways to control the behavior of the free learned attributes.The plan of this paper is the following: Sect. 2 presents the VarNet architecture together with its training algorithm. For better clarity, we introduce separately all the components featured in our model and postpone the discussion about their interplay and the motivation behind our modeling choices in Sect. 3 and Sect. 4 discusses about the related works. In particular , we show that VarNet provides an interesting solution to many constrained generation problems already considered in the literature. Finally, we illustrate in Appendix A the possibilities offered by our proposed model and show that its faculty to generate variations in an intended way is of particular interest. We presented the Variation Network, a generative model able to vary attributes of a given input. The novelty is that these attributes can be fixed or learned and have a sound probabilistic interpretation. Many sampling schemes have been presented together with a detailed discussion and examples. We hope that the flexibility in the design of the attribute function and the simplicity, from an implementation point of view, in transforming existing encoder-decoder architectures (it suffices to provide the encoder and decoder networks) will be of interest in many applications.For future work, we would like to extend our approach in two different ways: being able to deal with partially-given fixed attributes and handling discrete free attributes. We also want to investigate the of use stochastic attribute functions φ. Indeed, it appeared to us that using deterministic attribute functions was crucial and we would like to go deeper in the understanding of the interplay between all VarNet components.
The Variation Network is a generative model able to learn high-level attributes without supervision that can then be used for controlled input manipulation.
{"src": "scitldr-AIC_train", "paper_id": "ryfaViR9YX"}