Distributed Online System Identification for LTI Systems Using Reverse Experience Replay
Identification of linear time-invariant (LTI) systems plays an important role
in control and reinforcement learning. Both asymptotic and finite-time offline
system identification are well-studied in the literature. For online system
identification, the idea of stochastic-gradient descent with reverse experience
replay (SGD-RER) was recently proposed, where the data sequence is stored in
several buffers and the stochastic-gradient descent (SGD) update performs
backward in each buffer to break the time dependency between data points.
Inspired by this work, we study distributed online system identification of LTI
systems over a multi-agent network. We consider agents as identical LTI
systems, and the network goal is to jointly estimate the system parameters by
leveraging the communication between agents. We propose DSGD-RER, a distributed
variant of the SGD-RER algorithm, and theoretically characterize the
improvement of the estimation error with respect to the network size. Our
numerical experiments certify the reduction of estimation error as the network
size grows.