Spatial Big Data and Moving Objects: A Comprehensive Survey <s> 3) SPARSITY LEARNING <s> SUMMARY We propose a new method for estimation in linear models. The 'lasso' minimizes the residual sum of squares subject to the sum of the absolute value of the coefficients being less than a constant. Because of the nature of this constraint it tends to produce some coefficients that are exactly 0 and hence gives interpretable models. Our simulation studies suggest that the lasso enjoys some of the favourable properties of both subset selection and ridge regression. It produces interpretable models like subset selection and exhibits the stability of ridge regression. There is also an interesting relationship with recent work in adaptive function estimation by Donoho and Johnstone. The lasso idea is quite general and can be applied in a variety of statistical models: extensions to generalized regression models and tree-based models are briefly described. <s> BIB001 </s> Spatial Big Data and Moving Objects: A Comprehensive Survey <s> 3) SPARSITY LEARNING <s> Summary. We consider the problem of selecting grouped variables (factors) for accurate prediction in regression. Such a problem arises naturally in many practical situations with the multifactor analysis-of-variance problem as the most important and well-known example. Instead of selecting factors by stepwise backward elimination, we focus on the accuracy of estimation and consider extensions of the lasso, the LARS algorithm and the non-negative garrotte for factor selection. The lasso, the LARS algorithm and the non-negative garrotte are recently proposed regression methods that can be used to select individual variables. We study and propose efficient algorithms for the extensions of these methods for factor selection and show that these extensions give superior performance to the traditional stepwise backward elimination method in factor selection problems. We study the similarities and the differences between these methods. Simulations and real examples are used to illustrate the methods. <s> BIB002 </s> Spatial Big Data and Moving Objects: A Comprehensive Survey <s> 3) SPARSITY LEARNING <s> We derive generalizations of AdaBoost and related gradient-based coordinate descent methods that incorporate sparsity-promoting penalties for the norm of the predictor that is being learned. The end result is a family of coordinate descent algorithms that integrate forward feature induction and back-pruning through regularization and give an automatic stopping criterion for feature induction. We study penalties based on the l1, l2, and l∞ norms of the predictor and introduce mixed-norm penalties that build upon the initial penalties. The mixed-norm regularizers facilitate structural sparsity in parameter space, which is a useful property in multiclass prediction and other related tasks. We report empirical results that demonstrate the power of our approach in building accurate and structurally sparse models. <s> BIB003 </s> Spatial Big Data and Moving Objects: A Comprehensive Survey <s> 3) SPARSITY LEARNING <s> This paper investigates a new learning formulation called structured sparsity, which is a natural extension of the standard sparsity concept in statistical learning and compressive sensing. By allowing arbitrary structures on the feature set, this concept generalizes the group sparsity idea. A general theory is developed for learning with structured sparsity, based on the notion of coding complexity associated with the structure. Moreover, a structured greedy algorithm is proposed to efficiently solve the structured sparsity problem. Experiments demonstrate the advantage of structured sparsity over standard sparsity. <s> BIB004
Sparse learning is different from dimension reduction in a sense that it focuses on the removal of certain redundant features (such as dimension reduction of interference data) while preserving the originality of the remaining feature BIB001 ,  . Tibshirani BIB001 identifies these redundant features and give them extremely low weightage for ensuring smooth removal. Several refinements to this procedure are proposed later in  and BIB002 . Duchi and Singer BIB003 utilize sparsity penalization into the boosting algorithms for attaining higher performance. Although, there are several methods for employing the coding complexity related to the structure of the featured set BIB004 , it is still an important concern to differentiate redundant features and the features crucial for the performance.