A related question is that of the inductive bias. In over-parameterized settings, optimization methods, such as commonly used SGD and Adam [1]},
select a specific point \({\mathbf {w}}^*\)  in the set of parameters \({\mathcal {S}}\)  corresponding to interpolating solutions. In fact, given that \({\mathbf {w}}^*\)  depends on the initialization typically chosen randomly, e.g., from a normal distribution,
we should view \({\mathbf {w}}^*\)  as sampled from some induced probability distribution \(\mu \)  on the subset of \({\mathcal {S}}\)  reachable by optimization.
