We study the problem of estimating multiple linear regression equations for the purpose of both prediction and variable selection. Following recent work on multi-task learning [2008], we assume that the regression vectors share the same sparsity pattern. This means that the set of relevant predictor variables is the same across the different equations. This assumption leads us to consider the Group Lasso as a candidate estimation method. We show that this estimator enjoys nice sparsity oracle inequalities and variable selection properties. The results hold under a certain restricted eigenvalue condition and a coherence condition on the design matrix, which naturally extend recent work in [2007], Lounici [2008]. In particular, in the multi-task learning scenario, in which the number of tasks can grow, we are able to remove completely the effect of the number of predictor variables in the bounds. Finally, we show how our results can be extended to more general noise distributions, of which we only require the variance to be finite.
In order to study the statistical properties of this estimator, it is useful to derive the optimality condition for a solution of the problem . Since the objective function in is convex, @math is a solution of if and only if @math (the @math -dimensional zero vector) belongs to the subdifferential of the objective function. In turn, this condition is equivalent to the requirement that @math where @math denotes the subdifferential (see, for example, @cite_4 for more information on convex analysis). Note that Thus, @math is a solution of if and only if
@cite_4
1509803206
Background * Inequality constraints * Fenchel duality * Convex analysis * Special cases * Nonsmooth optimization * The Karush-Kuhn-Tucker Theorem * Fixed points * Postscript: infinite versus finite dimensions * List of results and notation.