Optimizing Training Trajectories in Variational Autoencoders via Latent Bayesian Optimization Approach
Unsupervised and semi-supervised ML methods such as variational autoencoders
(VAE) have become widely adopted across multiple areas of physics, chemistry,
and materials sciences due to their capability in disentangling representations
and ability to find latent manifolds for classification and regression of
complex experimental data. Like other ML problems, VAEs require hyperparameter
tuning, e.g., balancing the Kullback Leibler (KL) and reconstruction terms.
However, the training process and resulting manifold topology and connectivity
depend not only on hyperparameters, but also their evolution during training.
Because of the inefficiency of exhaustive search in a high-dimensional
hyperparameter space for the expensive to train models, here we explored a
latent Bayesian optimization (zBO) approach for the hyperparameter trajectory
optimization for the unsupervised and semi-supervised ML and demonstrate for
joint-VAE with rotational invariances. We demonstrate an application of this
method for finding joint discrete and continuous rotationally invariant
representations for MNIST and experimental data of a plasmonic nanoparticles
material system. The performance of the proposed approach has been discussed
extensively, where it allows for any high dimensional hyperparameter tuning or
trajectory optimization of other ML models.