Practically, each agent has a replay buffer that maintains a batch of trajectory data containing messages received during training to compute the CACL loss. Similar to [1]}, messages are normalized before the loss computation and a low temperature (i.e. \(\tau = 0.1\) ) is used which empirically shows benefits in performance. Together with the reinforcement learning (RL) loss \(L_{RL}\) , the total loss is formulated as follows:
\(L = L_{RL} + \kappa L_{CACL}\) 
