fig:computationalcomparison show the time and memory requirement for different approximation methods as a function of input size for a 5 layer convolutional network that preserve channel and input dimension. As baselines we use efficient implementations of the exact and kfac approximation [1]}, [2]}. The exact diagonal approximation run out of memory for an \(\sim 36 \times 36 \times 3\)  image on a 11 Gb NVIDIA GeForce GTX 1080 Ti. In contrast, our approximate diagonal Hessian scales linearly with the resolution, which is especially beneficial for convolutional layers.
