Table REF  presents the average test error evaluated on \(10\,000\)  images for the different AG-FNN configurations for \(\mathtt {E}=1\)  epoch, and for the optimal number of epochs found using early stop. In order to factor in the effect of random weight initialization, the results reported are the average and standard deviations from five runs. The performance achieved with respect to the average classification errors matches the reported state-of-the-art results of approximately 1.6% for FNNs having a same architecture with 2 layers and 800 hidden units and trained using gradient backpropagation [1]}, [2]}. We can see by comparing the results for 100 and 800 hidden units that increasing the hidden layer's size improves the performance both for the first and the optimal number of epochs. Moreover, the results in Table REF  indicate that TAGI's classification accuracy is not significantly affected by the usage of batch sizes greater than one. Nevertheless, we noticed though our experiments that using large batch sizes makes the learning phase sensitive to the network initialization as well as the observation noise parameter \(\sigma _{V}\) .
<TABLE>