In statistical inference problems, we wish to obtain lower bounds on the minimax risk, that is to bound the performance of any possible estimator. A standard technique to obtain risk lower bounds involves the use of Fano's inequality. In an information-theoretic setting, it is known that Fano's inequality typically does not give a sharp converse result (error lower bound) for channel coding problems. Moreover, recent work has shown that an argument based on binary hypothesis testing gives tighter results. We adapt this technique to the statistical setting, and argue that Fano's inequality can always be replaced by this approach to obtain tighter lower bounds that can be easily computed and are asymptotically sharp. We illustrate our technique in three applications: density estimation, active learning of a binary classifier, and compressed sensing, obtaining tighter risk lower bounds in each case.
Birg ' e @cite_20 gives stronger, but less transparent, bounds than Fano's inequality using Fano-type arguments; again, @math is bounded in terms of an average of Kullback--Leibler divergences, but these are used as the argument for a function, rather than directly substituted. Sason and Verd 'u [Section 3] sasonV18arimoto recently derived a generalized Fano's inequality in terms of the Arimoto-R 'enyi conditional entropy. They also obtained upper bounds on @math in terms of the pairwise R 'enyi divergences [Section 4] sasonV18arimoto .
@cite_20
2101412426
Lower bounds involving f-divergences between the underlying probability measures are proved for the minimax risk in estimation problems. Our proofs just use simple convexity facts. Special cases and straightforward corollaries of our bounds include well known inequalities for establishing minimax lower bounds such as Fano's inequality, Pinsker's inequality and inequalities based on global entropy conditions. Two applications are provided: a new minimax lower bound for the reconstruction of convex bodies from noisy support function measurements and a different proof of a recent minimax lower bound for the estimation of a covariance matrix.