For the experiments with BERT, we used the implementation from Contextualized Embeddings for Document Ranking (CEDR)https://github.com/Georgetown-IR-Lab/cedr provided by [1]}. We employed BERT base model (12 layers) with multilingual weights as well as a (Dutch) monolingual version called BERTje [2]}, which is trained on a large and diverse dataset of 2.4 billion tokens. We conducted separate experiments with both sets of weights in this work. We also added gradient clipping and warm-up steps from the HuggingFace transformer [3]} implementation to improve the performance.https://github.com/huggingface/transformers
