A multitude of rigorous analyses of AdaGrad, Adam and other adaptive methods have appeared in recent literature, notably [1]}, [2]}, [3]}. However, fully understanding the theory and utility of adaptive methods remains an active research area, with diverse (and sometimes clashing) philosophies [4]}, [5]}, [6]}.
