Due to its importance in machine learning, the KL divergence (and thus, the cross-entropy loss) is one of the few non-MSE losses for which the bias-variance tradeoff has been specifically analyzed [1]}, [2]}. For the KL divergence, the “central predictor” corresponds to an average in log-probability space, which has been studied in many works, including [3]}, [4]}. The fact that the bias remains unchanged when averaging predictions in log-probability space has been mentioned briefly in [5]}.
