A critical component of learning parsimonious models is in choosing hyperparameters that appropriately tradeoff model fit with sparsity, since adding more degrees of freedom monotonically decreases insample error. We strike this balance by selecting parameters which minimize the Akaike information criterion (AIC) metric [1]}, [2]}.
