Credit Assignment in RL. Credit assignment [1]}, [2]} tackles long-horizon sequential decision-making problem by distributing the contribution of each single step over the temporal interval. TD learning [3]} is the most established credit assignment method, which is the basis of many RL methods. RUDDER [4]} redistributes the episodic return to key timesteps in the episode [5]}, [6]}, [7]}. Klissarov et al. [8]} propose a reward propogation method via graph convolutional neural network [9]}. Another line of works utilize episodic memory (EM) [10]}, [11]}, [12]}, [13]}, [14]} to recall key events and aggregate information of the past for decision-making or learning. However, simply applying EM of RL to MARL cannot perform well in OBMAS due to the non-stationarity and the displaced rewards.
