This paper develops a theory for group Lasso using a concept called strong group sparsity. Our result shows that group Lasso is superior to standard Lasso for strongly group-sparse signals. This provides a convincing theoretical justification for using group sparse regularization when the underlying group structure is consistent with the data. Moreover, the theory predicts some limitations of the group Lasso formulation that are confirmed by simulation studies.
Finally, we shall mention that independent of the authors, results similar to those presented in this paper have also been obtained in @cite_1 with a similar technical analysis. However, while our paper studies the general group Lasso formulation, only the special case of multi-task learning is considered in @cite_1 .
@cite_1
2739284222
The Group-Lasso is a well-known tool for joint regularization in machine learning methods. While the l1,2 and the l1,∞ version have been studied in detail and efficient algorithms exist, there are still open questions regarding other l1,p variants. We characterize conditions for solutions of the l1,p Group-Lasso for all p-norms with 1 ≤ p ≤ ∞, and we present a unified active set algorithm. For all p-norms, a highly efficient projected gradient algorithm is presented. This new algorithm enables us to compare the prediction performance of many variants of the Group-Lasso in a multi-task learning setting, where the aim is to solve many learning problems in parallel which are coupled via the Group-Lasso constraint. We conduct large-scale experiments on synthetic data and on two real-world data sets. In accordance with theoretical characterizations of the different norms we observe that the weak-coupling norms with p between 1.5 and 2 consistently outperform the strong-coupling norms with p ≫ 2.