where \(\mathbb {E}_{\pi }[\cdot ]\)  is the (conditional) expectation when the policy \(\pi \)  is applied to the system
and \(\mathbf {z}(0) = \big (z_1(0),\ldots ,z_K(0)\big )\)  is the initial belief-stateWe assume that all policies \(\pi \)  induce a Markov chain with a single recurrent class plus a (possibly empty) set of transient states. Consequently, the minimum average cost is independent of the initial state [1]}.. We aim to find an optimal policy \(\pi ^\star \)  that achieves the minimum average cost, i.e.,
\((\textbf {P1})~~\pi ^\star \in ~ \textstyle {\operatornamewithlimits{arg\,min}_{\pi }}~\bar{C}_{\pi }.\) 
