The first instance of inferential methods playing a prominent role in fundamental physics occurred with the work of E.T. Jaynes in statistical mechanics [1]}. Up to that point, the development of statistical mechanics — from Boltzmann to Maxwell to Gibbs and so on — had been plagued by general confusion, as was evident with Maxwell's demon, the Gibbs paradox, irreversibility, etc.See, for example, the writings of Jaynes himself [2]}[3]} as well as the monograph by A. Caticha [4]} for an in-depth discussions of these issues and how the inference viewpoint casts light on them. The beauty of Jaynes' approach was that by placing the notions of probability, entropy, and information at the forefront of the theory, many of these fundamental issues and paradoxes in statistical mechanics were clarified, while many of the enigmatic building blocks of the conventional formulation were left behind. Indeed, once one adopts, as Jaynes did, a Bayesian view of probability, conceptual issues that hindered statistical mechanics — such as the ergodic hypothesis, etc. — were no longer relevant! The fuzzy picture of entropy as a measure of disorder, or chaos or (enter the interpretation du jour), of a system's physical state that was inherited from thermodynamics was replaced by entropy as a tool for processing information; a notion that began with the work of C. Shannon [5]}.
