Error Analysis of Gaussian Interpolation Is there any theory available on how well I can approximate a function using a basis of Gaussians 
$$
\phi_k(x) = \exp(-\tfrac{x - x_k}{\lambda_k})
?
$$
I've heard a bit about Gaussian process regression/Kriging, which seems related, but the approach in the related literatur is very statistical and hence not what I am interested in. 

Yes, it's called approximation by radial basis functions. See Wendland's Scattered Data Approximation or Fasshauer's Meshfree Approximation Methods With MATLAB.
