Recently, [1]} improves previous results to a more refined second-order bound \(\tilde{O}(\sqrt{\sum _{\tau \in I}\Vert \nabla \hspace{-1.38885pt}_{\tau }\Vert ^2})\) , but in a more restricted setting assuming the loss is linear.
