Many models based on the Variational Autoencoder are proposed to achieve disentangled latent variables in inference. However, most current work is focusing on designing powerful disentangling regularizers, while the given number of dimensions for the latent representation at initialization could severely inﬂuence the disentanglement. Thus, a pruning mechanism is introduced, aiming at automatically seeking for the intrinsic dimension of the data while promoting disentangled representations. The proposed method is validated on MPI3D and MNIST to be advancing state-of-the-art methods in disentanglement, reconstruction, and robustness. The code is provided on the https://github.com/WeyShi/FYP-of-Disentanglement. To advance disentanglement, models based on the Variational Autoencoder (VAE) (Kingma and Welling, 2014) are proposed in terms of additional disentangling regularizers. However, in this paper, we introduce an orthogonal mechanism that is applicable to most state-of-theart models, resulting in higher disentanglement and robustness for model configurationsespecially the choice of dimensionality for the latent representation. Intuitively, both excessive and deficient latent dimensions can be detrimental to achieving the best disentangled latent representations. For excessive dimensions, powerful disentangling regularizers, like the β-VAE (Higgins et al., 2017) , can force information to be split across dimensions, resulting in capturing incomplete features. On the other hand, having too few dimensions inevitably leads to an entangled representation, such that each dimension could capture enough information for the subsequent reconstruction. A pruning mechanism that is complementary to most current state-of-the-art VAE-based disentangling models is introduced and validated on MPI3D and MNIST. The approximated L 0 regularization facilitates the model to capture better-disentangled representations with optimal size and increases the robustness to initialization. Moreover, with the same hyperparameters, the model approaches the intrinsic dimension for several datasets including MNIST and MPI3D, even with an extra-large number of dimensions at initialization. Even given the intrinsic dimension, the PVAE still outperforms other SOTA methods in terms of disentanglement and reconstruction.
The Pruning VAE is proposed to search for disentangled variables with intrinsic dimension.
{"src": "scitldr-AIC_train", "paper_id": "HJg8stY2oB"}