Computing the hessian at every iteration vs. every epoch: We update the hessian estimate every iteration. In practice, [1]} updates the hessian every epoch. In principle, they could update the hessian more regularly, but to the best of our knowledge, they do not explore this path.
