Experimentally, however, it will be demonstrated that \( \beta -\) RUMSLR SGD yields noticeable improvements in minimization performance for datasets such as CIFAR-10, and CIFAR-100 ([1]}) when compared to SGD without stochastic learning rate. This scheme was equipped to the learning rates of other algorithms as well such as ADAM [2]}, AMSGrad [3]}, ADAMW [4]}, and two SGD with momentum algorithms
originally presented in [5]} and [6]} (subsequently referred to as SGD with momentum first and second versions respectively),
all of which, even though not theoretically studied in this work, yielded similar improved performance when using \( \beta -\) RUMSLR.
