Even though the aforementioned nonlinear modeling methods are appealing and powerful in providing analytic expressions for models, they are often built upon model hypotheses. For example, the success of sparse regression techniques relies on the fact that the nonlinear basis functions, describing the dynamics, lies in a candidate features library. For many complex dynamics, such as the melting Arctic ice, the utilization of these methods is not trivial. Thus, machine learning techniques, particularly deep learning-based ones, have emerged as powerful methods capable of expressing any complex function in a black-box manner given enough training data. Neural network-based approaches in the context of dynamical systems have been discussed in [1]}, [2]}, [3]}, [4]} decades ago. A particular type of neural networks, namely recurrent neural networks, intrinsically models sequences and is often used for forecasting [5]}, [6]}, [7]}, [8]}, [9]}. Deep learning is also utilized to identify a coordinate transformation so that the dynamics in the transformed coordinates are almost linear or sparse in a high-dimensional feature basis, see, e.g., [10]}, [11]}, [12]}, [13]}. Furthermore, we mention that classical numerical schemes are incorporated with feed-forward neural networks to have discrete-time steppers for predictions, see [3]}, [15]}, [16]}, [17]}. The approaches in [3]}, [15]} can be interpreted as nonlinear autoregressive models [20]}.
A crucial feature of deep learning-based approaches that integrates numerical integration schemes is that vector fields are estimated using neural networks. Also, time-stepping is done using a numerical integration scheme. However, measurement data are often corrupted with noise, and these mentioned approaches do not perform any specific noise treatment. The work in [21]} proposes a framework that explicitly incorporates the noise into a numerical time-stepping method. Though the approach has shown promising directions, its scalability remains ambiguous as the approach explicitly needs noise estimates and aims to decompose the signal explicitly into noise and ground truth.
