A state-of-the-art generative model, a ”factorized action variational autoencoder (FAVAE),” is presented for learning disentangled and interpretable representations from sequential data via the information bottleneck without supervision. The purpose of disentangled representation learning is to obtain interpretable and transferable representations from data. We focused on the disentangled representation of sequential data because there is a wide range of potential applications if disentanglement representation is extended to sequential data such as video, speech, and stock price data. Sequential data is characterized by dynamic factors and static factors: dynamic factors are time-dependent, and static factors are independent of time. Previous works succeed in disentangling static factors and dynamic factors by explicitly modeling the priors of latent variables to distinguish between static and dynamic factors. However, this model can not disentangle representations between dynamic factors, such as disentangling ”picking” and ”throwing” in robotic tasks. In this paper, we propose new model that can disentangle multiple dynamic factors. Since our method does not require modeling priors, it is capable of disentangling ”between” dynamic factors. In experiments, we show that FAVAE can extract the disentangled dynamic factors. Representation learning is one of the most fundamental problems in machine learning. A real world data distribution can be regarded as a low-dimensional manifold in a high-dimensional space BID3 . Generative models in deep learning, such as the variational autoencoder (VAE) BID25 and the generative adversarial network (GAN) BID15 , are able to learn low-dimensional manifold representation (factor) as a latent variable. The factors are fundamental components such as position, color, and degree of smiling in an image of a human face BID27 . Disentangled representation is defined as a single factor being represented by a single latent variable BID3 . Thus, if in a model of learned disentangled representation, shifting one latent variable while leaving the others fixed generates data showing that only the corresponding factor was changed. This is called latent traversals (a good demonstration of which was given by BID17 1 ). There are two advantages of disentangled representation. First, latent variables are interpretable. Second, the disentangled representation is generalizable and robust against adversarial attacks BID1 .We focus on the disentangled representation learning of sequential data. Sequential data is characterized by dynamic factors and static factors: dynamic factors are time dependent, and static factors are independent of time. With disentangled representation learning from sequential data, we should be able to extract dynamic factors that cannot be extracted by disentangled representation learning models for non-sequential data such as β-VAE BID17 b) and InfoGAN BID8 . The concept of disentangled representation learning for sequential data is illustrated in Fig. 1 . Consider that the pseudo-dataset of the movement of a submarine has a dynamic factor: the trajectory shape. The disentangled representation learning model for sequential data can extract this shape. On the other hand, since the disentangled representation learning model for non-sequential data does not consider the sequence of data, it merely extracts the x-position and y-position. Figure 1: Illustration of how FAVAE differs from β-VAE. β-VAE does not accept data sequentially; it cannot differentiate data points from different trajectories or sequences of data points. FAVAE considers a sequence of data points, taking all data points in a trajectory as one datum. For example, for a pseudo-dataset representing the trajectory of a submarine (1a,1c), β-VAE accepts 11 different positions of the submarine as non-sequential data while FAVAE accepts three different trajectories of the submarine as sequential data. Therefore, the latent variable in β-VAE learns only the coordinates of the submarine, and the latent traversal shows the change in the submarines position. On the other hand, FAVAE learns the factor that controls the trajectory of the submarine, so the latent traversal shows the change in the submarines trajectory.There is a wide range of potential applications if we extend disentanglement representation to sequential data such as speech, video, and stock market data. For example, disentangled representation learning for stock price data can extract the fundamental trend of a given stock price. Another application is the reduction of action space in reinforcement learning. Extracting dynamic factors would enable the generation of macro-actions BID11 , which are sets of sequential actions that represent the fundamental factors of the actions. Thus, disentangled representation learning for sequential data opens the door to new areas of research.Very recent related work BID22 BID26 ) separated factors of sequential data into dynamic and static factors. The factorized hierarchical variational autoencoder (FHVAE ) BID22 ) is based on a graphical model using latent variables with different time dependencies. By maximizing the variational lower bound of the graphical model, the FHVAE separates the different time dependent factors such as the dynamic and static factors. The VAE architecture developed by BID26 is the same as the FHVAE in terms of the time dependencies of the latent variables. Since these models require different time dependencies for the latent variables, these approaches cannot be used disentangle variables with the same time dependency factor.We address this problem by taking a different approach. First, we analyze the root cause of disentanglement from the perspective of information theory. As a result, the term causing disentanglement is derived from a more fundamental rule: reduce the mutual dependence between the input and output of an encoder while keeping the reconstruction of the data. This is called the information bottleneck (IB) principle. We naturally extend this principle to sequential data from the relationship between x and z to x t:T and z. This enables the separation of multiple dynamic factors as a consequence of information compression. It is difficult to learn a disentangled representation of sequential data since not only the feature space but also the time space should be compressed. We created the factorized action variational autoencoder (FAVAE) in which we implemented the concept of information capacity to stabilize learning and a ladder network to learn a disentangled representation in accordance with the level of data abstraction. Since our model is a more general model without the restriction of a graphical model design to distinguish between static and dynamic factors, it can separate depen-dency factors occurring at the same time. Moreover, it can separate factors into dynamic and static factors.2 DISENTANGLEMENT FOR NON-SEQUENTIAL DATA β-VAE BID17 b) is a commonly used method for learning disentangled representations based on the VAE framework BID25 ) for a generative model. The VAE can estimate the probability density from data x. The objective function of the VAE maximizes the evidence lower bound (ELBO) of log p (x) as DISPLAYFORM0 where z is latent variable, D KL is the Kullback-Leibler divergence, and q (z|x) is an approximated distribution of p (z|x). D KL (q (z|x) ||p (z|x)) reduces to zero as the ELBO L VAE increases; thus, q (z|x ) learns a good approximation of p (z|x). The ELBO is defined as DISPLAYFORM1 where the first term, E q(z|x) [log p (x|z)], is a reconstruction term used to reconstruct x, and the second term D KL (q (z|x) ||p (z)) is a regularization term used to regularize posterior q (z|x). Encoder q (z|x) and decoder p (x|z) are learned in the VAE.Next we will explain how β-VAE extracts disentangled representations from unlabeled data. β-VAE is an extension of the coefficient β > 1 of the regularization term DISPLAYFORM2 where β > 1 and p (z) = N (0, 1). β-VAE promotes disentangled representation learning via the Kullback-Leibler divergence term. As β increases, the latent variable q (z|x) approaches the prior p (z) ; therefore, each z i is pressured to learn the probability distribution of N (0, 1). However, if all latent variables z i become N (0, 1), the model cannot reconstruct x. As a result, as long as z reconstructs x, β-VAE reduces the information of z.
We propose new model that can disentangle multiple dynamic factors in sequential data
{"src": "scitldr-AIC_train", "paper_id": "Hygm8jC9FQ"}