We are interested in computing the hessian of a loss function. For this purpose, the Jacobian of the nn w.r.t. parameters plays a crucial role. In this section we develop a better understanding of this object, we derive the backpropagation (also used by the BackPack library [1]}) and finally we explain our approximated backpropagation that allows linear scaling.
