This work introduces the concept of parametric Gaussian processes (PGPs), which is built upon the seemingly self-contradictory idea of making Gaussian processes parametric. Parametric Gaussian processes, by construction, are designed to operate in "big data" regimes where one is interested in quantifying the uncertainty associated with noisy data. The proposed methodology circumvents the well-established need for stochastic variational inference, a scalable algorithm for approximating posterior distributions. The effectiveness of the proposed approach is demonstrated using an illustrative example with simulated data and a benchmark dataset in the airline industry with approximately 6 million records.
Despite some subtle differences, it is generally safe to recognize the input-output pairs @math (see equation )) as the so called inducing points'', a frequently used term in the literature on sparse approximations to Gaussian process priors (see e.g., @cite_3 for a compressive review). However, it is not advisable to interpret @math and @math (see equation )) as variational parameters @cite_5 since no (stochastic) variational inference is carried out in the current work. Furthermore, to highlight the subtle differences between inducing points'' and what this work calls ), it is worth observing that in the literature on sparse approximations to Gaussian processes it turns out that @math and @math . Under these assumptions and using equations ) and ), one obtains @math and @math . In other words, in the sparse Gaussian processes framework, @math and @math are essentially identical; i.e., @math . In contrast, this work treats @math and @math as parameters of the model responsible for encoding the history of observed data. In this regard, the current work is similar to @cite_5 . However, unlike @cite_5 , the parameter @math and @math are not variational parameters of some variational distribution.
@cite_5 @cite_3
2952677397 1571870753
We introduce stochastic variational inference for Gaussian process models. This enables the application of Gaussian process (GP) models to data sets containing millions of data points. We show how GPs can be vari- ationally decomposed to depend on a set of globally relevant inducing variables which factorize the model in the necessary manner to perform variational inference. Our ap- proach is readily extended to models with non-Gaussian likelihoods and latent variable models based around Gaussian processes. We demonstrate the approach on a simple toy problem and two real world data sets. We provide a new unifying view, including all existing proper probabilistic sparse approximations for Gaussian process regression. Our approach relies on expressing the effective prior which the methods are using. This allows new insights to be gained, and highlights the relationship between existing methods. It also allows for a clear theoretically justified ranking of the closeness of the known approximations to the corresponding full GPs. Finally we point directly to designs of new better sparse approximations, combining the best of the existing strategies, within attractive computational constraints.