Sentence Meta Embeddings for Unsupervised Semantic Textual Similarity
We address the task of unsupervised Semantic Textual Similarity (STS) by ensembling diverse pre trained sentence encoders into sentence meta embeddings. We apply, extend and evaluate different meta embedding methods from the word embedding literature at the sentence level, including dimensionality reduction (Yin and Schutze, 2016) generalized Canonical Correlation Analysis (Rastogi et al. 2015) and cross view auto encoders (Bollegala and Bao, 2018) Our sentence meta embeddings set a new unsupervised State of The Art (SoTA) on the STS Benchmark and on the STS12 STS16 datasets, with gains of between 3.7% and 6.4% Pearson's r over single source systems.
ACL