The model parameters \((\theta _{s}, \theta _{r})\)  are trained on \((Q, \mathcal {H}, \hat{Q})\)  pairs. Since the spans \(Z\)  are not available in training data, the REINFORCE algorithm [1]} is used for model training. Let \(R(Z, \hat{Q}^*)\)  be the function (or reward) to measure the quality of \(Z\)  with respect to the reference \(\hat{Q}^*\)  as
\(R(Z, \hat{Q}^*) = \sum _{\hat{Q} \in \mathcal {\hat{Q}}} P_r(\hat{Q}|Z ; \theta _{r}) r(\hat{Q},\hat{Q}^{*}),\) 
