We have used the sequential GP model fitting via AL is another approach to improve the tractability of (exact) GP regression for large datasets. Essentially, we choose the most relevant subset of the training data set, instead of using the entire data set or randomly sampling from it. fig:rmse shows the log RMSE of the GP—based on a held out test data set of 1781 points—with sequential increments in the training data. Recall that the points are chosen sequentially to reduce the average uncertainty about the GP in the entire input domain \(\mathcal {X}\) , and fig:rmse shows the resulting prediction error (log RMSE) variation with training data set size. We start the GP fit with a randomly selected 50 points (from the full data set of 5000 points) and sequentially add 100 more points. We also show the impact of the choice of the number of points selected (\(q\) ) at each step by varying it between \(q=1\)  to \(q=8\) . We repeat this exercise 20 times for independently chosen random starting points and plot the mean and \(\pm 1\)  standard deviation. Finally, we also show the prediction error due to selecting 250 points randomly in one shot (i.e., no sequential point selection). The results show that sequential point selection results in a smaller prediction error, regardless of the choice of \(q\) , for the first 3 latent space dimensions. For the fourth latent space dimension (\(\tilde{\mathbf {x}}_4\) ), the \(q=1\)  still outperforms the one-shot selection. It is worth noting that the one-shot selection of points still has 100 points more than what was supplied to the AL GP. The reason for the AL GP outperforming the one-shot GP is because training points are more judiciously chosen–in this case, they are chosen specifically to minimize the uncertainty in the GP about its own prediction. Furthermore, it can be shown [1]} that the average uncertainty in the GP is equivalent to the mean-squared prediction error (MSPE) of the posterior mean of the GP, and therefore in effect the AL chooses points to minimize the overall prediction error.
<FIGURE>