To mitigate the problem, Chung et al. [1]} proposed Hierarchical Multiscale Recurrent Neural Network (HM-RNN). HM-RNN is able to learn the temporal boundaries from data, which facilitates the RNN model to build a hierarchical structure and enable long-term dependencies automatically. However, the temporal boundaries are stochastic discrete variables which are very hard to train using standard back propagation algorithm.
