Generally, adding depth to an ANN by increasing the number of layers, or increasing the number of neurons/nodes, improves its accuracy. This has been reported in various papers [1]}, [2]}, [3]}, [4]}, where the authors experimented with different numbers of layers and found that deeper ANNs gave more accurate predictions. Similar findings were reported in [5]} and [6]}, where the authors observed that increasing the number of neurons while keeping the number of hidden layers constant, improved the accuracy of the RBF-ANN. Bigger ANNs (and generally more complex models) are more prone to overfitting, especially if only limited numbers of training data are available. Hence, regularization techniques have to be used, such as early stopping or L1 and L2 regularization [7]}, [8]}. The former method stops network training when the validation error follows an increasing trend, a sign of overfitting. The latter penalize big network weights by incorporating the L1 and L2 weight norm, respectively, into the cost function (see Eq. (REF ) in the Appendix).
