We recall that in the ordinary learning problem, the bound for generalization gap can be obtained for average generalization error scenario [1]}, [2]}, [3]}, [4]} and PAC-Bayes scenario [5]}, [6]}, [7]}, [8]}, [9]}, [10]}, [11]}, [12]}, [13]}, [14]}, [15]}.
In the former case, the bound of generalization error is derived by averaging over the training set and hypothesis. While, the PAC-Bayes bounds hold with high probability.
