In order to enable the results presented in Section REF  and render value iteration easy to solve, the bounds \(\check{P}(\cdot ,a,\cdot )\)  and \(\hat{P}(\cdot ,a,\cdot )\)  have to be (piecewise) linear or concave/convex or convex/concave on the action variables. For instance, this is the case for a large class of switching diffusion models where the continuous action controls the probability of switching between different modes of the system [1]}. In more general cases, the functions \(\inf _{x\in q_i}\)  and \(\sup _{x\in q_i}\mathbb {P}_w(f(x,a,w) \in q_j)\)  may need to be lower-/upper-bounded via (piecewise) linear, convex or concave functions.
