Long Short-Term Memory (LSTM) [1]} solves this problem by introducing three gates: input gate, forget gate, output gate. They are integrated together to capture spatial dependencies for a long time by keeping the gradient in the same order of magnitude during back-propagation.
The hidden state \(h_{t}\)  depends not only on the current input \(x_{t}\) , but also previous time step hidden state \(h_{t-1}\) .
