language: en
tags:
- sentence-embeddings
- sentence-similarity
mirror-bert-base-uncased-sentence-drophead
An unsupervised sentence encoder proposed by Liu et al. (2021), using drophead instead of dropout as feature space augmentation. Trained with unlabelled raw sentences, using bert-base-uncased as the base model. Please use mean-pooling over all tokens as the representation of the input.
Citation
@inproceedings{
liu2021fast,
title={Fast, Effective and Self-Supervised: Transforming Masked LanguageModels into Universal Lexical and Sentence Encoders},
author={Liu, Fangyu and Vuli{\'c}, Ivan and Korhonen, Anna and Collier, Nigel},
booktitle={EMNLP 2021},
year={2021}
}