Another line of work deals with mean-field residual networks by looking at the continuum limit of residual networks when either the depth \(L\)  or the width \(d\)  goes to infinity. [1]} build on the analysis of [2]} for feed-forward networks to study the average behaviour of randomly initialized residual networks with width tending to infinity. They show that a careful initialization, depending on the depth, may enhance expressivity. Further, [3]} proposed a continuum limit of deep residual networks by letting the depth \(L\)  tends to infinity and showed that every local minimum of the loss landscape is global. This characterization enables them to derive the first global convergence result for multi-layer neural networks in the mean-field regime.
