The bias-variance tradeoff has been an important tool in understanding the behavior of machine learning models [1]}, [2]}, [3]}, [4]}, [5]}, [6]}, [7]}. Key to the decomposition are the notions of a “central label” (if label noise exists) — and of a “central prediction”. Most analyses focus on the bias-variance decomposition for the Euclidean square loss; in this case, the central label and prediction correspond respectively to the mean label and prediction. [8]} proposed a more general decomposition for symmetric losses, [9]} focussed on the 0-1 loss, and [10]} proposed a generalization to the space of all Bregman divergences. [11]} identify which loss functions admit bias-variance decompositions with specific properties (but do not analyze the decompositions themselves). [12]}, [13]} decompose twice differentiable losses via second order Taylor expansion. In the general case — including for the KL divergence — the resulting decompositions are approximate. Additionally, the variance term may depend on target labels, in a significant departure from any standard definition of a variance.
