In the differential regression learning method, we use a combination of the loss \(L^*_{val}\)  and \(L^*_{der}\)  in the training of the neural networks for approximating the value function and its derivative. We train alternately the value and the derivative of the network by taking a gradient step to minimize \(L^*_{val}\)  every even number of epochs and a gradient step to minimize \(L^*_{der}\)  every odd number of epochs. An alternative method would be to minimize a convex combination of \(L^*_{val}\)  and \(L^*_{der}\)  with weights \(w_{val}\)  and \(w_{der} = 1 - w_{val}\) . These weights could be chosen by performing a grid search or a random search, such as advocated in [1]} for the choice of neural network hyperparameters. Some theoretical arguments could also be derived in order to choose these weights, as it has been done in [2]} to optimally choose the weights of the losses associated to different constraints of a PDE when using Physics Informed Neural Networks [3]}. Our approach proves to be effective, as shown by the numerical results in Sections REF  and REF , and avoids the need to chose a value for this additional hyperparameter. The algorithmic implementation and the pseudo-codes are described in Section REF .
