As we have shown in Section , the training of large-scale RNNs also need to compute the sample-path gradient of the sample total loss with respect to the weights of the neural networks, which is typically done by the BP algorithms. The BP algorithm for training neural networks was proposed by [1]}, and it is one of the corner stones of deep learning. The idea of the BP algorithm is to compute the gradient using a reverse mode, after finishing the calculation of the function value. It is different from the IPA algorithm, which uses a forward mode to calculate the gradient alongside the calculation of the function value. It has been shown that, when the input is of a multi-dimensional vector (such as the weight vector in RNN training or the base-stock level vector in inventory optimization), the BP algorithm is typically computationally more efficient than the forward mode algorithms (see, for instance, [2]}).
