Considerable research has been done in evaluating the effect of learning rate on model performance [1]}, [2]}, [3]} and how they relate to each other. A study conducted by Popel et al. demonstrated that small learning rates tend to have slower convergence, whereas learning rates that are too high may lead to non-convergence [4]}.
