We can see that the KL loss of VAE-LSTMbase, which uses Sigmoid annealing <cite>(Bowman et al., 2016)</cite> , collapses to zero, leading to a poor generative performance as indicated by the high reconstruction loss.