For the biencoder model (BEM), we use the codebase provided by [1]}. Following this work, we train the BEM for 20 epochs with a warmup phase of 10,000 steps; we use a context batch size of 4 and a gloss batch size of 256. Each BEM is tuned over the following learning rates: \([1e-6, 5e-6, 1e-5, 5e-5]\) . The BEM\(_{BERT}\)  and BEM\(_{SemCor}\)  had a final learning rate of \(5e-6\) , and the BEM\(_{FEWS}\) , of \(1e-6\) .
