To solve Problem (REF ), SGD algorithms are typically used. Notice that SGD algorithms are originated from SA algorithms, and they have been adapted to solve very high-dimensional problems [1]}. To use SGD algorithms, one needs to calculate the gradient of \(L_T^n\)  with respect to the network parameters. This is typically done through BP algorithm. The BP algorithm for calculating the gradient of neural networks is a celebrated result in the history of deep learning [2]}. It is particularly efficient if the network parameters are of high dimension, and it may be implemented automatically [3]}. The particular version of the BP algorithm for RNNs is called back-propagation through time [4]}, and it is capable of handling neural networks with a few million parameters [5]}.
