Deep Gaussian Process [1]} (DGP) is a hierarchical composition of GPs.
Theoretical progresses regarding understanding DGPs have been made via several important observations. In the deep limit, DGPs collapse to a constant function for some subspace of hyperparameters [2]}, [3]}, [4]} and carry a heavy-tailed distribution over function derivatives [2]}. [6]} showed that the covariance and kurtosis are analytical characteristics of some two-layer DGPs, and a similar transition into chaotic phase with heavy-tailed multivariate statistics.
