I am studying neural networks. The smaller we make the learning rate, the longer the memory span over which the LMS algorithm remembers past data will be. (Section 3.5(page 103) of Simon Hykin's Neural Network and Learning Machines:)I don't understand why this is the case. Could someone explain?
Learning rate and memory of Least Mean Square algorithm
neural network;algorithms