The largest obstacle to apply la in practice stem from the Hessian matrix. This matrix has a quadratic memory complexity in the number of network parameters, which very quickly exceeds the capabilities of available hardware.
To counter this issue, several approximations have been proposed [1]}, [2]}, [3]} that improve the scaling w.r.t. to the number parameters.
The currently most efficient Hessian implementations [4]}, [5]} builds on the generalized Gauss-Newton (ggn) approximation of the Hessian
\(\nabla ^2_{{\theta }_l} \mathcal {L}(f_{\theta }(\mathbf {x}))\approx J_{{\theta }_l}f_{\theta }\left(\mathbf {x}\right)^{\!\top } \cdot \nabla ^2_{\mathbf {x}_\text{rec}}\mathcal {L}\left(\mathbf {x}_\text{rec}\right) \cdot J_{{\theta }_l}f_{\theta }\left(\mathbf {x}\right),\) 
