Remark 2 Note that, while in this work we only consider cumulative rewards properties, similarly to IMDPs with discrete action spaces, our results can easily be extended to more general properties, such as bounded-time fragments of PCTL [1]} or LTL [2]}. Infinite-horizon (unbounded-time) extensions are also possible, but particular care is needed to guarantee the convergence of the value iteration in Proposition REF  due to the non-finitness of \(\mathcal {A}\) .

