Masked Region Prediction (MRP). Similar to the MLM, we randomly mask out proposal inputs \(p_{v}^{m}\)  with approximately 15%. Different from the discrete work token, in the task, we directly regress the masked features \(p_i\) , which is supervised by the features extracted by the bottom-up-attention model [1]} with an MSELoss. The masked proposal inputs \(h_t^m\)  are fed into a fully-connected (FC) layer (\(FC_{MRM}\) ) and project to a binary space for determining whether the input is masked :
\(\begin{aligned}p_{i}^{\prime } &=\mathrm {FC}_{\mathrm {MRM}}\left(p_{v}^{\mathrm {m}}\right), \\\mathcal {L}_{\mathrm {MRM}} &=\frac{1}{2\left|\mathcal {M}^{\mathrm {MRM}}\right|} \sum _{i \in \mathcal {M}^{\mathrm {MRM}}} \left|p_{i}^{\prime } - p_{i}^{t} \right|^2,\end{aligned}\) 
