[1]} introduced and [2]} improved deep kernel learning.
The inputs to a classic GP kernel \(k\)  (e.g. RBF) are preprocessed by
applying a feature extractor \(g\)  (a deep NN) prior to computing the kernel:
\(k_\text{deep}({\bf {X}}, {\bf {X}}^{\prime }) :=k(g({\bf {X}}; \theta ), g({\bf {X}}^{\prime }, \theta ))\) .
The NN parameters are optimised by gradient ascent using the likelihood as the objective, as in standard GP kernel learning [3]}.
Since deep kernel learning incorporates a state-of-the-art NN with over
\(10^6\)  parameters, we expect it to perform similarly to a NN applied
directly to the task of image classification. At present both CNN and deep
kernel learning
display superior performance to the GP kernels in this work.
However, the kernels defined here have far fewer parameters (around 10,
compared to their \(10^6\) ).
