Can I bound the correlation of two random variables using the mutual information? As correlation
$\rho_{X,Y} := \frac{Cov(X,Y)}{\sigma_X \sigma_Y}$
sort of measures the linear dependence of two random variables, and mutual information
$I(X; Y) := H(X) - H(X|Y)$
measures the general dependence of two random variables, I feel like it should be possible to get an upper bound on the correlation in terms of the mutual information.
I expect that as the mutual information increases, correlation tends to 1, and as mutual information tends to 0, correlation also does.
Can anyone help me formalise this?

This answer is physicist-oriented, but it should answer your question! It has the infinity norm in the denominator, I don't know if you can do better.
Suppose we have sets $A$ and $B$, joint probability distribution $p_{ab}$, and individual probability distributions $p_a=\sum_b p_{ab}$. We have the identities:


*

*$I(A:B)=D(p_{ab} \| p_a p_b)$ (a property of Kullback-Leibler)

*$D(p_a\|q_a)\ge \frac{1}{2}\left(\sum_{a\in A} |p_a-q_a|\right)^2$ (Pinsker's inequality, proof here)


Then, given functions $f(a)=f_a$ and $g(b)=g_b$, we may write:
$$I(A:B)\ge \frac{1}{2}\frac{\left(\langle f g\rangle-\langle f\rangle \langle g\rangle \right)^2}{\|f\|_\infty^2 \|g\|_\infty^2}$$
Proof:
\begin{align}
I(A:B)&=D(p_{ab} \| p_a p_b)\\
&\ge\frac{1}{2}\left(\sum_{ab} |p_{ab}-p_a p_b|\right)^2\\
&\ge \frac{1}{2}\left(\sum_{ab} \Big|(p_{ab}-p_a p_b)\frac{f_a g_b}{\|f\|_\infty\|g\|_\infty}\Big|\right)^2\\
&=\frac{1}{2}\frac{\left(\langle f g\rangle-\langle f\rangle \langle g\rangle \right)^2}{\|f\|_\infty^2 \|g\|_\infty^2} 
\end{align}
(multiplying by numbers $0\le x\le 1$ is free) $$\tag*{$\square$}$$

physics citations: 
p.4 of this paper by Wolf et al 
p.36 of these notes by McGreevy
