So when should one use test log-likelihood?
We argue that comparing probabilistic forecasts using test log-likelihood is reasonable when either (a) the predictive distribution is Gaussian (so maximizing \(\textrm {lpd}\)  ensures correct mean and variance forecasts) or when (b) being close to the true data generating distribution in a Kullback-Leibler sense is substantively important.
It is important to note, however, that just because two distributions are close in KL, their means and variances need not be close; in fact, Propositions 3.1 & 3.2 of [1]} show that the means and variances of distributions that are close in KL can be arbitrarily far apart.
