This paper is concerned with the robustness of VAEs to adversarial attacks. We highlight that conventional VAEs are brittle under attack but that methods recently introduced for disentanglement such as @math -TCVAE (, 2018) improve robustness, as demonstrated through a variety of previously proposed adversarial attacks ( (2016); Gondim- (2018); (2018)). This motivated us to develop Seatbelt-VAE, a new hierarchical disentangled VAE that is designed to be significantly more robust to adversarial attacks than existing approaches, while retaining high quality reconstructions.
This is the hierarchical version of the collapse of @math units in a single-layer VAE @cite_17 , but now the collapse is over entire layers @math . It is part of the motivation for the Ladder VAE @cite_1 and BIVA @cite_35 . 2pt
@cite_35 @cite_1 @cite_17
1995406764 2915002151 2038198231 2047042737
For @math -dimensional tensors with possibly large @math , an hierarchical data structure, called the Tree-Tucker format, is presented as an alternative to the canonical decomposition. It has asymptotically the same (and often even smaller) number of representation parameters and viable stability properties. The approach involves a recursive construction described by a tree with the leafs corresponding to the Tucker decompositions of three-dimensional tensors, and is based on a sequence of SVDs for the recursively obtained unfolding matrices and on the auxiliary dimensions added to the initial “spatial” dimensions. It is shown how this format can be applied to the problem of multidimensional convolution. Convincing numerical examples are given. Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. In this paper, we investigate several multi-level structures to learn a VAE model to generate long, and coherent text. In particular, we use a hierarchy of stochastic layers between the encoder and decoder networks to generate more informative latent codes. We also investigate a multi-level decoder structure to learn a coherent long-term structure by generating intermediate sentence representations as high-level plan vectors. Empirical results demonstrate that a multi-level VAE model produces more coherent and less repetitive long text compared to the standard VAE models and can further mitigate the posterior-collapse issue. We define the hierarchical singular value decomposition (SVD) for tensors of order @math . This hierarchical SVD has properties like the matrix SVD (and collapses to the SVD in @math ), and we prove these. In particular, one can find low rank (almost) best approximations in a hierarchical format ( @math -Tucker) which requires only @math parameters, where @math is the order of the tensor, @math the size of the modes, and @math the (hierarchical) rank. The @math -Tucker format is a specialization of the Tucker format and it contains as a special case all (canonical) rank @math tensors. Based on this new concept of a hierarchical SVD we present algorithms for hierarchical tensor calculations allowing for a rigorous error analysis. The complexity of the truncation (finding lower rank approximations to hierarchical rank @math tensors) is in @math and the attainable accuracy is just 2-3 digits less than machine precision. Let (B s , s≥ 0) be a standard Brownian motion and T1 its first passage time at level 1. For every t≥ 0, we consider ladder time set ℒ (t) of the Brownian motion with drift t, B (t) s = B s + ts, and the decreasing sequence F(t) = (F1(t), F2(t), …) of lengths of the intervals of the random partition of [0, T1] induced by ℒ (t) . The main result of this work is that (F(t), t≥ 0) is a fragmentation process, in the sense that for 0 ≤t < t′, F(t′) is obtained from F(t) by breaking randomly into pieces each component of F(t) according to a law that only depends on the length of this component, and independently of the others. We identify the fragmentation law with the one that appears in the construction of the standard additive coalescent by Aldous and Pitman [3].