Efficient Transformed Gaussian Processes for Non-Stationary Dependent Multi-class Classification
This work introduces the Efficient Transformed Gaussian Process (ETGP), a new
way of creating C stochastic processes characterized by: 1) the C processes are
non-stationary, 2) the C processes are dependent by construction without
needing a mixing matrix, 3) training and making predictions is very efficient
since the number of Gaussian Processes (GP) operations (e.g. inverting the
inducing point's covariance matrix) do not depend on the number of processes.
This makes the ETGP particularly suited for multi-class problems with a very
large number of classes, which are the problems studied in this work. ETGPs
exploit the recently proposed Transformed Gaussian Process (TGP), a stochastic
process specified by transforming a Gaussian Process using an invertible
transformation. However, unlike TGPs, ETGPs are constructed by transforming a
single sample from a GP using C invertible transformations. We derive an
efficient sparse variational inference algorithm for the proposed model and
demonstrate its utility in 5 classification tasks which include
low/medium/large datasets and a different number of classes, ranging from just
a few to hundreds. Our results show that ETGPs, in general, outperform
state-of-the-art methods for multi-class classification based on GPs, and have
a lower computational cost (around one order of magnitude smaller).