We implement CASe based on the BERT implementation in PyTorch by Hugging Face,
using the base-uncased pre-trained model with 12 layers and 768-dim hidden state. The maximum input length \(m\)  is 512 in which the maximum query length is 40. The random sampling dimension \(d_R\)  is 768. The input dimension of the first layer in the adversarial network is 768. And its intermediate dimension is 512, using ReLU as the activation function in first two layers. Generating probability threshold \(T_{prob}\)  is set as 0.4 and \(n_{best}=20\) . Adam optimizer [1]} is employed with learning rate \(3\times 10^{-5}\)  in the source domain training, \(2\times 10^{-5}\)  in the self-training and \(10^{-5}\)  in the adversarial learning, with batch size 12. A dropout with rate 0.2 is applied on both the BERT feature network and the discriminator. We set the epoch number \(N_{pre}=3\)  in pre-training and \(N_{da}=4\)  in domain adaptation.
