Our probe and BEM baselines are in implemented in PyTorchhttps://pytorch.org/ and optimized with Adam [1]}. For the BEM, we use the implementation provided by [2]}.https://github.com/facebookresearch/wsd-biencoders We obtain the bert-base-uncased encoder from [3]} to get the BERT output representations for the probe and to initialize the BEM models. Further hyperparameter details are given in Appendix .
