Variational autoencoder-based data augmentation (VAE-DA) is a domain adaptation method proposed in<cite> [13]</cite> , which pools in-domain and out-domain to train a VAE that learns factorized latent representations of speech segments. In this paper, we extend VAE-DA and address the issue by learning interpretable and disentangled representations using a variant of VAEs that is designed for sequential data, named factorized hierarchical variational autoencoders (FHVAEs) [15] .