RNN is a class of neural networks that allow previous outputs to be used as inputs while having hidden states. This enables it to exhibit temporal dynamic behaviors and to model time series data. RNN models have achieved the-state-of-the-art performance on tasks that include speech recognition [1]}, language modeling [2]}, and image captioning [3]}. The scale of deep RNN models can be very large, for example, the model of [1]} contains 8.5 million parameters.
