Equipped with directed cycles in network connections, RNN retains information from past data, so that it can be combined with the present data to more accurately predict the future sequence of data.
However, RNN suffers from the vanishing or exploding gradient problem during training [1]}. If the RNN is not designed properly, the gradient may be exponentially updated towards zero, referred to as a vanishing gradient, or exponentially diverge to infinity, referred to as an exploding gradient. Either of these problems challenges the proper RNN training.
To resolve this problem, GRU is equipped with a specific mechanism to determine the volume of past data, to be added to the output. By doing so, the problem of vanishing or exploding gradient is eliminated. As illustrated in Fig. REF , GRU consists of three elements: the reset gate, the update gate, and the current memory unit [2]}. The update gate determines the volume of past data that are used for predicting the future data sequence, while the reset gate determines the volume of the stored past information to be forgotten. The current memory unit uses the output from the reset gate to store the relevant information from the past data. The output of GRU is the sum of the output from the three gates.
The three gates and the final output can be described as,
\(r_{t} = \sigma ( W^{r}x_{t} + U^{t}h_{t-1} )\\z_{t} = \sigma ( W^{z}x_{t} + U^{z}h_{t-1} )\\h_{t}^{^{\prime }} = \tanh (Wx_{t} + r_{t}\odot Uh_{t-1})\\h_{t} = z_{t}\odot h_{t-1} + (1-z_{t})\odot h_{t}^{^{\prime }} .\) 
