Then, there are some works study the implicit bias of GD on deep neural networks. The asymptotic direction of the product of weights in a deep linear network is studied in [1]}, [2]}. [3]} studied the implicit bias of two-layer neural network with ReLU activation. [4]} proved the asymptotic direction is along a KKT point of the \(L^2\)  max margin problem for homogeneous deep neural networks. [5]} independently proved similar result for homogeneous neural networks with simplified assumptions. Based on [5]}, [7]} further prove that parameters have only one asymptotic direction.
