For our experiments, we follow the training and evaluation procedure outlined in <cite>Akbik et al. (2018)</cite> and follow most hyperparameter suggestions as given by the in-depth study presented in Reimers and Gurevych (2017) .