We now focus on the variance. Using the fact that \(D_{F}[p\Vert  q] = D_{F^*}[q^*\Vert  p^*]\)  [1]}, we have
\(\mathbb {E}D_F(\mathcal {E}\hat{X} \Vert  \hat{X}) &= \mathbb {E}D_F[\mathcal {E}X \Vert  \hat{X}] \\&= \mathbb {E}D_{F^*}[\hat{X}^* \Vert  (\mathcal {E}X)^*] \\&= \mathbb {E}D_{F^*}\Big [\frac{1}{n} \sum \nolimits _i X_i^* \Vert  (\mathcal {E}X)^*\Big ] \\&\overset{(a)}{\le } \frac{1}{n} \sum \nolimits _i \mathbb {E}D_{F^*}\Big [X_i^* \Vert  (\mathcal {E}X)^*\Big ] \\&\le \frac{1}{n} \sum \nolimits _i \mathbb {E}D_{F}\Big [\mathcal {E}X \Vert  X_i\Big ] \\&\le D_F[\mathcal {E}X \Vert  X].\) 
