
It is known that a pair of stationary policies \((x_\star , y_\star )\)  attaining the minimax value on state \(s\)  is necessarily attaining the minimax value on all states [1]},
and we call such \(x_\star \)  a minimax policy, such \(y_\star \)  a maximin policy, and such pair a Nash equilibrium.
Further define \(\mathcal {X}^s_{\star }=\lbrace x^s_{\star } \in x_{\star } : \text{$x_{\star }$ is a minimax policy}\rbrace \)  and similarly \(\mathcal {Y}^s_{\star }=\lbrace y^s_{\star } \in y_{\star } : \text{$y_{\star }$ is a maximin policy} \rbrace \) , and denote \(\mathcal {Z}_\star ^s = \mathcal {X}^s_{\star } \times \mathcal {Y}^s_{\star }\) .
It is also known that any \(x= \lbrace x^s\rbrace _{s\in \mathcal {S}}\)  with \(x^s \in \mathcal {X}^s_{\star }\)  for all \(s\)  is a minimax policy (similarly for \(y\) ) [1]}.
