HM-RNN was proposed in [1]} to better capture the hierarchical multi-scale temporal structure in sequence modeling. HM-RNN defines three operations depending on the boundary detectors: UPDATE, COPY and FLUSH. The selection of these operations is determined by boundary state \(z_t^{l-1}\)  and \(z_{t-1}^l\) , where \(l\)  and \(t\)  stand for the current layer and time step, respectively:
\(\begin{array}{ll}UPDATE, & \hbox{$z_{t-1}^l = 0$ and $z_t^{l-1} = 1$;} \\COPY, & \hbox{$z_{t-1}^l = 0 $ and $z_t^{l-1} = 0$;} \\FLUSH, & \hbox{$z_{t-1}^l = 1$.}\end{array}\) 
