Alternatively, we may construct a system model using data following a system identification or machine learning approach. The corresponding system model can often be written in the form of an artificial neural network, where the type of neurons can range from commonly used neurons with sigmoidal or rectified linear activation functions to wavelets, splines, radial basis functions, monomials or Gaussian processes, to name a few [1]}. Generally designed to be a universal function approximator, the use of an artificial neural network often leads to an unnecessarily complex model. High complexity strongly hampers the use of the model in real-time applications, and it commonly goes hand in hand with stringent data requirements. It should be noted that the complexity of the model strongly depends on the chosen coordinate frame and type of neurons. As a first example, the orbits of the planets in our solar system can be described much simpler and in fewer terms by taking the Sun as center of reference than Earth. As mentioned in [2]} and [3]}, many physical systems allow for a sparse representation in a suitable coordinate frame. As a second example, it takes a large artificial network with rectified linear units to accurately approximate a quadratic function on a large (but finite) domain, while it requires only three monomials to do the same (i.e., monomials of degrees zero, one, and two, respectively). Training such a large artificial network needs many measurements of the quadratic function, while fitting the three monomials requires a minimum of only three data points (disregarding measurement noise). Without any knowledge of the system to go by, the chance that a chosen coordinate frame and neuron type result in an accurate model of low complexity is generally slim.
