Recurrent neural network (RNN) enables recurrent connections among neurons.
RNN learns a representation for each time point that takes into account the
previous input history and the current input. Unlike HMM, which ignored
earlier points beyond a threshold, RNNs do not constrain the memory length
[1]}. In RNN, the far earlier inputs give less influence
to the representation of the current input. This implicit fading memory has
enabled RNN a powerful tool in language modeling [2]}, sequence process [3]} and video
prediction [4]} etc.
