Following the initial work of McAllester [1]}, PAC-Bayes bounds for conventional learning have been widely investigated. Selecting different convex functions, which link the expected and empirical losses, such as KL-divergence [2]}, square function [3]} or linear function [4]} implies different PAC-Bayes bounds.
The dependency on the sample size, in most of these bounds, is inversely proportional to the square root of the number of samples.
In [5]}, by choosing the convex function as \(\mathrm {D}_{\gamma }(a||b)=\gamma a-\log (1-b+be^\gamma )\) , a family of PAC-Bayes bounds known as fast-rate bounds were obtained.
In these kinds of bounds, the dependence on the sample size can be improved by the inverse of the number of samples. Directly relevant to this paper, in [6]} by proposing a general approach of finding PAC-Bayes bounds, various known and also new PAC-Bayes bounds were obtained.
