Generalization.  Traditional VC dimension-based generalization bounds depend on the number of parameters and are vacuous for huge models such as overparameterized neural networks. To overcome this limitation, several non-vacuous generalization bounds are proposed. For example, the norm/margin-based generalization bounds [1]}, [2]}, [3]}, [4]}, and the PAC-Bayes-based bounds [5]}, [6]}, [7]}, [8]}.
However, these bounds tend to focus less on optimization, e.g., norm-based generalization bounds may not discuss how small-norm solutions are obtained through practical training.
In this paper, we connect the optimization and generalization by deriving generalization bounds based on the optimization path length.
