IPPO exploits centralised training solely through sharing network parameters between agent critics. However, SMAC additionally provides full state information that lifts partial observability of the environment during learning.
While at first glance, conditioning the critic on full state information should help simply due to it containing more information, already early empirical research in independent vs centralised learning concludes that, perhaps counter-intuitively, this is not necessarily the case as the larger size of the full state and/or it including information irrelevant to the task may adversely impact learning [1]}. Nevertheless, previous work on SMAC have reported higher performance when the centralised state was being exploited during learning[2]}, [3]}
A direct comparison (see Table REF ) shows that replacing IPPO's local critics with parameter-sharing critics conditioning on full state information performs substantially worse on a selection of hard SMAC maps, calibrating our implementation with centralised value function performance stated in concurrent work [3]}.
<FIGURE>