A question that naturally arises is whether to enforce sparsity as a hard constraint or promote sparsity as an objective penalty. While it may be tempting to use a penalty term and let the model balance sparsity, it is known that constrained problems enjoy more favorable statistical properties [1]}. In particular, while an optimal solution to the sparsity regularized problem is always obtainable by the constrained problem, the converse is not true in general (see [2]} Section 2.2). This is especially true when the data matrix exhibits high multicollinearity which is common in system discovery because the library terms are usually strongly correlated.
