Minimum-variance control of adaptive optics (AO) systems relies on a stochastic dynamical model of the perturbation and on models of the components, including loop delays. Resulting LQG controllers have been implemented in SCAO and WFAO both on laboratory benches and on-sky. Their efficiency has been recognized in several modes of operation, e.g. i) on-sky control of TT or low-order modes with vibration mitigation (SPHERE, GPI, CANARY, Raven, GeMS, in H2 formulation at the McMath-Pierce solar telescope) ii) full SCAO mode (CANARY) and MOAO mode (CANARY, Raven) and iv) in general it is advocated to control the low-order modes in laser tomography systems (E-ELT HARMONI LTAO, NFIRAOS). We first point out two examples related to VLT AO controllers to illustrate the need for RTC flexibility. The implementation of LQG control in the framework of the future ELTs raises many questions related both to real-time control computation and associated parameter updates (at a far lower rate), and to the performance that can be reached compared with simpler control strategies. By gathering many lab and on-sky results, we draw the performance trends observed so far. We then outline some promising research directions for control design and implementations for future ELTs AO systems.