First, we emphasize the difference is in the second order term. Similarly to [1]} and [2]}, we use the diagonal of the hessian (technically the diagonal of the GGN), while Adam uses the pointwise square of the gradient, which they call second raw moment and is intended as a cheaply computable approximation of the hessian.
