We computed the robust optimal policy and expected cumulative reward \(\mathcal {R}_{\star }(q)\)  (for all \(q\) ) over the caIMDP, via convex programming, as instructed by Corollary REF . Moreover, we obtained policies and the corresponding rewards \(\mathcal {R}_s(q)\)  (for all \(q\) ) by performing synthesis over discrete-action IMDPs, which are discretized versions of the original caIMDP: a discrete-action IMDP is constructed as \(I^s_{discr}=(\mathcal {Q}, \hat{\mathcal {A}}_s, \check{P}, \hat{P}, R)\) , where \(\hat{\mathcal {A}}_s\subseteq \mathcal {A}\)  is a finite set of \(s\)  actions randomly sampled from the continuous action set \(\mathcal {A}\) ; that is, \(|\hat{\mathcal {A}}_s| = s\) . Specifically, we computed \(\mathcal {R}_s(q)\)  for different values of \(s\) . Note that, since randomness is involved in the experimentsIn the case of value-iteration over the caIMDP with convex programming, the initial condition for the optimization is picked randomly, which affects the total computation time. In the case of discrete-action IMDPs, the sampled actions affect both the computation time and, especially, the reward \(\mathcal {R}_s(q)\) ., we performed them multiple timesValue iteration over the caIMDP has been performed 3 times. The experiments on discrete-action IMDPs for \(s=1,8,\dots ,256\)  have been performed \([20, 11, 9, 6, 3]\)  times, respectively.. Moreover, to perform synthesis over the discrete-action IMDPs (i.e. to solve the \(s\)  associated LPs), we used the algorithm of [1]}. Finally, optimality tolerance for convex programming was set to \(10^{-4}\) .
<FIGURE>