Can I bound the correlation of two random variables using the mutual information? As correlation
$\rho_{X,Y} := \frac{Cov(X,Y)}{\sigma_X \sigma_Y}$
sort of measures the linear dependence of two random variables, and mutual information
$I(X; Y) := H(X) - H(X|Y)$
measures the general dependence of two random variables, I feel like it should be possible to get an upper bound on the correlation in terms of the mutual information.
I expect that as the mutual information increases, correlation tends to 1, and as mutual information tends to 0, correlation also does.
Can anyone help me formalise this?

For the mutual information, it can be useful to consider the conditional entropy instead:
$$H(X|Y) = H(X,Y) - H(X)$$
However, the claim in the question is incorrect, because correlation indicates only linear dependence, while mutual information relates to dependence in general.
Going back one step to covariance, we can find the following example:


*

*$Y = X^2, X$ uniform distributed in $[-1,1]$

*$\Rightarrow \sigma(X,Y) = 0, \sigma(X)= E(X^2)\neq 0, \sigma(Y)= E(X^4) \neq 0$, thus we get $\rho_{X,Y} = 0$

*However, as stated in conditional entropy: $H(Y|X) = 0$, because $Y$ is completely determined by the value of $X$.

*For the mutual information, we get: $I(X;Y) = H(Y) - H(Y|X)  = H(Y)$. And this does not tend to $0$, even if the correlation is $0$ already.

