First, as in [1]}, we define \(\mathcal {F}_{Q_E}\)  as the set of on-\(Q\)  moments, with \(f \in \mathcal {F}_{Q_E}\)  satisfying type signature \(\mathcal {S} \times \mathcal {A} \times \mathcal {C} \rightarrow [-T, T]\) . Intuitively, \(\mathcal {F}_{Q_E}\)  spans the set of possible expert \(Q\) -functions. We require the actual expert \(Q\) -function to be contained: \(Q^{\pi _E} \in \mathcal {F}_{Q_E}\) . We assume that \(\mathcal {F}_{Q_E}\)  is convex, compact, closed under negation, and finite dimensional. Second, define \(\mathcal {F}_{Q}\)  be the class of off-\(Q\)  moments (i.e. \(\forall \pi \in \Pi \) , \(Q^{\pi } \in \mathcal {F}_{Q}\) ) and satisfy the same assumptions as \(\mathcal {F}_{Q_E}\) . Third, let \(\mathcal {F}_{r}\)  denote the class of reward moments and also satisfy the same function-class assumptions. All \(f \in \mathcal {F}_{r}\)  satisfy type signature \(\mathcal {S} \times \mathcal {A} \times \mathcal {C} \rightarrow [-1, 1]\)  and we assume that \(r \in \mathcal {F}_r\) .
