Another fictitious play algorithm is proposed in [1]}, which in some ways parallels [2]} by iteratively updating both the policy and the mean-field term at every step of the algorithm. In the algorithm of [1]}, there is no nested loop structure, whereby the optimal policy is estimated iteratively while the mean-field term is held fixed. As with the algorithm of [4]}, the main algorithm of [1]} is not suitable for use when data arrives to the agent from an actual trajectory of play, as the main algorithm of [1]} requires the agent to “do nothing” if it so chooses (C.f. Line 4, [1]}).
