To obtain BERT\(_{\scriptsize \textrm {DAPT}}\) , we fine-tune bert-base-cased
(BERT\(_{\scriptsize \textrm {BASE}}\) ) using language modeling scripts in
Transformers [1]} for 3 epochs using default hyperparameters.
To clarify the effects of DAPT, we experiment with BERT\(_{\scriptsize \textrm {BASE}}\)  as well.
For CLF, we fine-tune BERT on task training data for 10 epochs with a learning rate of 5e-5, early stopping on the Target dev set.
All our \(k-\) NN experiments are done with with the \(k=100\)  using Faiss library [2]}.
