We apply TAGI to the 1D regression problem \(y=x^{3}+v\) , \(v:V\sim \mathcal {N}(0,9)\)  taken from [1]}, using an AG-FNN having a single hidden layer with 100 units, and a ReLU activation function. The objective of this case study is to showcase how TAGI can be applied on small datasets (\(\mathtt {D}=20\)  points), and to compare the results obtained by considering either diagonal or full covariance matrices. In this example, the inference is performed using one observation at a time, where both the covariates \(x\)  and observations \(y\)  were normalized in the range \([-1,1]\) . The optimal number of epochs is identified from a validation set \(\mathcal {D}_{\mathtt {V}}\)  consisting in 20 additional points. The prior covariance for bias is initialized to \(\Sigma _{{B}}^{0}=0.01\cdot \mathbf {I}\) , and for weights \(\Sigma _{{W}}^{0}\) , by multiplying the Xaviers's approach [2]} by a factor 0.25. The prior mean vector is randomly sampled from \({\mu }_{{\theta }}^{0}\sim \mathcal {N}(\mathbf {0},\Sigma _{{\theta }}^{0})\) .
