Where \(\displaystyle \eta \)  denotes the learning rate, \(\displaystyle \nabla {f_t}\left( {{\theta _t}} \right)\)  denotes the gradient of the objective function \(\displaystyle {f_t}\left( {{\theta _t}} \right)\)  of the \(\displaystyle t\) -th iteration with respect to the parameter \(\displaystyle {\theta _t}\) . The online learning[1]} is a commonly used theoretical analysis framework for the above models. In the online optimization problem, the loss function sequence is \(\displaystyle {f_1}\left( {{\theta _1}} \right),{f_2}\left( {{\theta _2}} \right),...,{f_t}\left( {{\theta _t}} \right),...,{f_T}\left( {{\theta _T}} \right)\) , where \(\displaystyle {\theta _t}\)  is the parameter that needs to be optimized. The goal of online learning is to minimize the sum of the difference between each iteration loss and the optimal loss[2]}. The theoretical analysis of this paper is under the framework of online learning.
