Although the aforementioned works show effectiveness in addressing the latent variable collapse issue to some extent, they either require carefully engineering to balance the weight between the reconstruction loss and KL loss (Bowman et al., 2016; , or resort to designing more sophisticated model structures (Yang et al., 2017; <cite>Xu and Durrett, 2018</cite>; Dieng et al., 2019) . In this paper, we present a simple architecture called holistic regularisation VAE (HR-VAE), which can effectively avoid latent variable collapse.