In this paper, we develop an efficient method for training an artificial neural network for system models described by ordinary differential equations. Common training algorithms include gradient descent, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) algorithm and the Levenberg-Marquardt algorithm [1]}, [2]}, [3]}. Compared to the first-order convergence of gradient descent, a faster second-order convergence can be obtained using L-BFGS or Levenberg-Marquardt. The Levenberg-Marquardt algorithm has the advantage that it is relatively easy to parallelize. Due to the Markovian nature of the system model, we can rewrite the approximation of the optimization problem by Levenberg and Marquardt as batches of recursive small-scale optimization subproblems. We effectively divide the time interval from the first to the last measurement in separate subintervals, where each subinterval corresponds to one batch of optimization subproblems. All batches can be solved in parallel. Subsequently, the solution to the original approximation can be reconstructed by combining the results from all batches. Parallelization can significantly reduce the wall time required to train the network. In addition, we outline how a similar problem formulation can be used to prune the network using backward elimination by removing irrelevant network edges one at a time.
