Ranking the Pivot Timesteps. With our two search schemes, we can search the pivot timesteps for each global trajectory \(\tau =\lbrace (s^{t}, \mathbf {\tilde{\text{$u$}}}^{t}, r^t, s^{t+1})\rbrace ^{T-1}_{t=0}\) . We define the pivot timesteps \(\kappa \)  of each global trajectory \(\tau \)  as
\(\kappa =\lbrace e_{t}\rbrace ^{T-1}_{t=0}, ~0 \le e_{t} \le t\) ,
where \(e_{t}\)  indicates the pivot timestep of \(t\)  when \(r_t\)  is the consequence of actions committed before timestep \(t\) . We first get \(e_{t}\)  by aggregating all the searching outcomes (Line REF  in Alg. REF ).
Then, each agent gets \(\kappa _i=\lbrace e^{i}_{t}\rbrace ^{T-1}_{t=0}\) . In order to subserve the inter-agent credit assignment [1]}, [2]}, \(\kappa \)  can be collectively calculated via proximity:

\(e_{t} = \min _{e^{i}_{t}}\left[ t-e^{1}_{t}, \cdots , t-e^{N}_{t}\right], ~ i \in \lbrace 1, \cdots , N\rbrace \) 
