The enabling technology underlying all of these methods is the optimization algorithm that selects and fits the best set of library terms to reconstruct the dynamics. The original SINDy paper [1]} used the sequential threshold least squares algorithm which finds a sparse solution by iteratively fitting a least squares regression on the candidate library and removing terms with coefficients below a specified threshold. However, thresholding is problematic for recovering small model coefficients and does not easily allow adding additional structure on the coefficients (e.g., enforcing physical symmetries). These limitations motivated the development of algorithms with different sparse regularizers that could incorporate constraints such as the sparse relaxed regularized regression (SR3) [2]}, [3]} and Conditional Gradients based approaches [4]}. There is also a long history of greedy algorithms and convex relaxations designed to solve sparse regression and related subset selection problem [5]}, [6]}, [7]}. For system identification problems, we claim that such relaxations and heuristics are unnecessary, and ultimately, inadequate.
