This training stage is repeated with various NN architectures to
find the optimal one capturing at best the functional dependence
of the data. Evaluation of the MSE is carried out through a stochastic
gradient descent minimization using the Adam optimization algorithm
[1]} giving a measure of the loss with a learning rate
starting at \(0.01\)  and reducing most of the time to \(0.0001\)  during
the training, \(\beta _1=0.9\) , \(\beta _2=0.999\)  and \(\varepsilon =10^{-8}\) .
The early-stopping was performed with maximum loss variation of
\(10^{-6}\)  and a patience of 45 epochs. The typical duration of
the training period was about 10000 to 15000 epochs. The least
MSE loss is obtained for an architecture of 10 neurons in the
first and second hidden layers. A typical evolution
of loss and the validation loss is shown in Fig. REF (a).
A cross-validation performed over 5 independent trainings gives
a RMSE of \((3.1\pm 0.5)\)  meV on the per atom energy. Figure REF (b)
displays the predictive ability of the model on the unseen data of
the test set, with a high quality over the whole range of the
energies.
<FIGURE>