We compare various Gaussian process priors with a random series prior using B-splines. B-splines are widely used in nonparametric regression [1]}, [2]}, and theoretical properties of using the B-spline prior with normal basis coefficients to estimate function derivatives have been recently studied in [3]}. The implementation of this B-spline prior follows [3]}. In particular, for any \(x\in [0,1]\) , let \( b_{J,4}(x)=(B_{j,4}(x))_{j=1}^J\)  be a B-spline of order 4 and degrees of freedom \(J\)  with uniform knots. The prior on \(f\)  is given as \(f(x)=b_{J,4}(x)^T \beta \)  with each entry of \(\beta \)  following \(N(0, \sigma ^2)\)  independently. The unknown variance \(\sigma ^2\)  is estimated by its MMLE \(\hat{\sigma }_n^2=n^{-1}Y^T(B B^T+\mathbf {I}_n)^{-1}Y\) , where \(B=(b_{J,4}(X_1),\ldots ,b_{J,4}(X_n))^T\) . The number of interior knots \(N=J-4\)  is determined using leave-one-out cross validation. We adopt the same strategy of leave-one-out cross validation to select the degree of freedom \(\nu \)  in \(K_{\text{Mat},\nu }\)  for a fair comparison. In particular, we consider \(\nu \)  from the set \(\lbrace 2, 2.5, 3, 3.5, 4\rbrace \)  and vary \(N\)  from 1 to 10. We also include a Matérn kernel with the oracle value \(\nu =2.5\)  that matches the smoothness level of \(f_0\) . For Gaussian process priors, the regularization parameter \(\lambda \)  and unknown \(\sigma ^2\)  are determined by empirical Bayes through maximizing the marginal likelihood.
