Given a node at level \(t\) , agents collectively search from the node to find the pivot time step (Line REF  in Alg. REF ). The visit count is vital for search methods. In MARL, we use \(\epsilon \) -greedy [1]} for agents to explore the environment and collect individual trajectories. The collected trajectories will be used to build the memory and train the policy. We apply annealing to \(\epsilon \)  (in Appx. ).
