A third approach of arriving at an acceptance criterion is by cross-validation. Although there are many different forms of cross-validation, the main principle behind cross-validation is that the measurement data is partitioned in a training set, that is solely used for training the neural network, and a validation set (and/or test set; see [1]}, for example), to evaluate how well the predictions of the trained network generalize to previously unseen data. In that case, a proposed removal of a network edge is only accepted if it results in a lower value of the cost function based on the validation data. Note that determining the cost function value for the validation set requires solving the optimization problem in (REF ), where the (fixed) values of the network parameters are determined in the training stage. The use of a validation set avoids the problem of selecting a suitable limit value for the cost function or a suitable information criterion. However, because the training set and validation set must each cover the relevant part of the state space, this may put an unreasonably large demand on the amount of available measurement data.
