We adopt Adam optimizer [1]} with a learning rate of 2e-5 for the transfer step on the ASNQ dataset and a learning rate of 1e-6 for the adapt step on the target dataset.
We apply early stopping on the dev. set of the target corpus for both steps based on the highest MAP score. We set the max number of epochs equal to 3 and 9 for adapt and transfer steps, respectively.
We set the maximum sequence length for BERT/RoBERTa to 128 tokens.
