Distributed Feature Selection for High-dimensional Additive Models
Distributed statistical learning is a common strategy for handling massive
data where we divide the learning task into multiple local machines and
aggregate the results afterward. However, most existing work considers the case
where the samples are divided. In this work, we propose a new algorithm,
DDAC-SpAM, that divides features under the high-dimensional sparse additive
model. The new algorithm contains three steps: divide, decorrelate, and
conquer. We show that after the decorrelation operation, every local estimator
can recover the sparsity pattern for each additive component consistently
without imposing strict constraints to the correlation structure among
variables. Theoretical analysis of the aggregated estimator and empirical
results on synthetic and real data illustrate that the DDAC-SpAM algorithm is
effective and competitive in fitting sparse additive models.