The proof of Theorem REF  is shown in Appendix A.3.
From Theorem REF , we know that the FMGDM algorithm should enjoy a linear convergence rate by choosing appropriate \( \alpha \)  and \( \gamma \) , as long as both \( n \)  and \( t \)  are large enough.
By Theorem REF  (a), we know that the optimal convergence factor \( \rho = (\sqrt{\lambda _{1}/\lambda _{p}} - 1) / (\sqrt{\lambda _{1}/\lambda _{p}} + 1)\)  can be achieved by carefully specified \( \alpha \)  and \( \gamma \) .
This specification is closely related to the eigenvalue structure of the covariance matrix \( \Sigma _{xx} \) .
Our findings are in line with theoretical findings in the classical GDM method without minibatch [1]}. It is known that this convergence factor also attains the achievable lower bound of various first-order methods [2]}.
Moreover, Theorem REF  (b) claims that, for a standard GD algorithm with \( \gamma =0 \) , the best convergence factor should be \( \rho = (\lambda _1/\lambda _{p} -1 ) / (\lambda _1/\lambda _{p} +1)\) .
This represents a convergence rate that is strictly slower than that of the FMGDM algorithm as long as \( \lambda _1>\lambda _p \) .
