To compare discrete distributions on a fixed grid (i.e. histograms), one can directly rely on usual norms such as the \(\ell _2\)  and the \(\ell _1\)  norms.
The \(\ell _1\)  norm is often preferable, because it can be extended to general measures (i.e. not necessarily discrete) to define the Total Variation (TV) norm. A related construction is the relative entropy (also called the Kullback-Leibler/KL divergence), which is fundamental in statistics and ML [1]} because of its connection to maximum likelihood estimation [2]}, [3]} and information geometry [4]}. Section REF  details these two discrepancies (KL and TV) within the more general framework of \(\phi \) -divergences.
