In the context of supervised classification, PAC-Bayesian bounds have proved to be the tightest [1]}, [2]}, [3]}. Several recent works have focused
on gradient descent based PAC-Bayesian algorithms,
aiming to minimise a generalisation bound for stochastic classifiers [4]}, [5]}, [6]}. Most of these studies use a surrogate loss to avoid dealing with the zero-gradient of the misclassification loss. Several authors used other methods to estimate of the misclassification error with a non-zero gradient by proposing new training algorithms to evaluate the optimal output distribution in PAC-Bayesian bounds analytically [7]}, [8]}, [9]}.
Recently, there have been some interesting works which use information-theoretic approach to find PAC-bounds on generalization errors for machine learning [10]}, [11]} and deep learning [12]}.
