Q:

Explanation of Parsimony

Can someone explain what Parsimony is in the context of probability, more specifically in Parsimonious Markov models?
I have been trying to search around a simple explanation of this but I only seem to be getting domain-specific papers in biology etc. which assume the reader already know what it means.

A:

Parsimony is basically Occam's razor: prefer simpler models when possible. In linear regression, for example, a model is parsimonious when it includes relatively few predictors, interactions, higher-order terms, etc. One motivation for parsimony is avoiding over-fitting in high-dimensional data.
Parsimony is a principle of modeling more than of probability (though you can see it as the natural result of having high prior probability on simple models and low prior probability on complex models), and not everybody agrees with it.

