DistSPECTRL: Distributing Specifications in Multi-Agent Reinforcement Learning Systems
While notable progress has been made in specifying and learning objectives
for general cyber-physical systems, applying these methods to distributed
multi-agent systems still pose significant challenges. Among these are the need
to (a) craft specification primitives that allow expression and interplay of
both local and global objectives, (b) tame explosion in the state and action
spaces to enable effective learning, and (c) minimize coordination frequency
and the set of engaged participants for global objectives. To address these
challenges, we propose a novel specification framework that allows natural
composition of local and global objectives used to guide training of a
multi-agent system. Our technique enables learning expressive policies that
allow agents to operate in a coordination-free manner for local objectives,
while using a decentralized communication protocol for enforcing global ones.
Experimental results support our claim that sophisticated multi-agent
distributed planning problems can be effectively realized using
specification-guided learning.