Kernel regression is an essential and ubiquitous tool for non-parametric data analysis, particularly popular among time series and spatial data. However, the central operation which is performed many times, evaluating a kernel on the data set, takes linear time. This is impractical for modern large data sets. In this paper we describe coresets for kernel regression: compressed data sets which can be used as proxy for the original data and have provably bounded worst case error. The size of the coresets are independent of the raw number of data points, rather they only depend on the error guarantee, and in some cases the size of domain and amount of smoothing. We evaluate our methods on very large time series and spatial data, and demonstrate that they incur negligible error, can be constructed extremely efficiently, and allow for great computational gains.
The only non-parametric regression coreset we are aware of is a form of kernel regression @cite_24 related to the smallest enclosing ball. It predicts the value at a point @math as @math with loss function @math , for a parameter @math . Then it finds a set of @math non-zero @math parameters (corresponding with points in the coreset) so many points satisfy @math . No implementations were attempted.
@cite_24
1497808028
A rigid coreset minimum enclosing ball training machine for kernel regression estimation was proposed. First, it transfers the kernel regression estimation machine problem into a center-constrained minimum enclosing ball representation form, and subsequently trains the kernel methods using the proposed MEB algorithm. The primal variables of the kernel methods are recovered via KKT conditions. Then, detailed theoretical analysis and main theoretical results of our new algorithm are given. It can be concluded that our proposed MEB training algorithm is independent of sample dimension and the time complexity is linear in sample numbers, which greatly cuts down the complexity level and is expected to speedup the learning process obviously. Finally, comments about the future development directions are discussed.