We modify the HuggingFace [1]} codebase for our experiments. We use the Adam [2]} optimizer with \((\beta _1,\beta _2)=(0.9,0.999)\)  and learning rate 5e-5. The model is trained for 25000 steps using batch size 4. We use the same article split for training, validation and testing as in DCQA, and the parameters are tuned on the validation set.
