Recent successes of value-based multi-agent deep reinforcement learning employ optimism by limiting underestimation updates of value function estimator, through carefully controlled learning rate (, 2017) or reduced update probability (, 2018). To achieve full cooperation when learning independently, an agent must estimate the state values contingent on having optimal teammates; therefore, value overestimation is frequency injected to counteract negative effects caused by unobservable teammate sub-optimal policies and explorations. Aiming to solve this issue through automatic scheduling, this paper introduces a decentralized quantile estimator, which we found empirically to be more stable, sample efficient and more likely to converge to the joint optimal policy.
This specification discloses a reduction gear transmission comprising an input shaft carrying a pinion, a satellite carrier having a gear meshing with the pinion, a composite satellite gear on said carrier and including two sections, one with a greater number of teeth than the other, an output gear meshing with the satellite gear section having the smaller number of teeth, an output shaft drivably carrying the output gear, and a holding gear meshing with the other section of the satellite gear and having a hub in which the output shaft is journalled. The hub extends through an opening in the housing in which the aforesaid gear mechanism is mounted. An infinite ratio gear assembly is operatively associated with the hub to control its rate of rotation. This gear assembly includes an inner pair of beveled gears in confronting relation, relatively axially movable, and keyed to the output shaft; an outer pair of beveled ring gears in confronting relation to each other and also confronting the inner beveled gears, a mechanical interlock to cause said outer beveled gears to rotate in unison, all of the faces of said beveled gears having radial grooves, a pin ring disposed in the space defined by the faces of the beveled gears, pins carried by said pin ring and having ends received in said grooves, and a ring shifting device to move the ring radially and thereby adjust the radial positions of the pin ends in the grooves.