How could we plot the KL divergence? I was trying to write a blogpost on information theory and I think it would be a good idea (if possible) to plot the KL divergence in a 3D-plot in order to show graphically its convexity, but I wouldn't know how to define the pdf space. How would you do it?
$$
KL(f||g)=\sum_{x \in X} f(x)\log \frac{f(x)}{g(x)}
$$

One idea would be to use the fact that a function is convex if and only if its restriction to a line is convex.
In the case of KL divergence, we can pick any two pairs of distributions $(f, g)$ and $(f', g')$ and plot
$$
\mathrm{KL}(\lambda f + (1 - \lambda) f' \, || \, \lambda g + (1 - \lambda) g')
$$
as a function of $\lambda$, with $\lambda \in [0, 1]$.
