Information divergence functions play a critical role in statistics and information theory. In this paper we show that a non-parametric f-divergence measure can be used to provide improved bounds on the minimum binary classification probability of error for the case when the training and test data are drawn from the same distribution and for the case where there exists some mismatch between training and test distributions. We confirm the theoretical results by designing feature selection algorithms using the criteria from these bounds and by evaluating the algorithms on a series of pathological speech classification tasks.
This paper presents new upper and lower bounds on the minimum probability of error of Bayesian decision systems for the two-class problem. These bounds can be made arbitrarily close to the exact minimum probability of error, making them tighter than any previously known bounds. In this paper, we present a new upper bound on the minimum probability of error of Bayesian decision systems for statistical pattern recognition. This new bound is continuous everywhere and is shown to be tighter than several existing bounds such as the Bhattacharyya and the Bayesian bounds. Numerical results are also presented. >