Bilingual lexicons map words in one language to their translations in another , and are typically induced by learning linear projections to align monolingual word embedding spaces . In this paper , we show it is possible to produce much higher quality lexicons with methods that combine ( 1 ) unsupervised bitext mining and ( 2 ) unsupervised word alignment . Directly applying a pipeline that uses recent algorithms for both subproblems significantly improves induced lexicon quality and further gains are possible by learning to filter the resulting lexical entries , with both unsupervised and semisupervised schemes . Our final model outperforms the state of the art on the BUCC 2020 shared task by 14 F 1 points averaged over 12 language pairs , while also providing a more interpretable approach that allows for rich reasoning of word meaning in context . Further analysis of our output and the standard reference lexicons suggests they are of comparable quality , and new benchmarks may be needed to measure further progress on this task . 1 Bilingual lexicons map words in one language to their translations in another , and can be automatically induced by learning linear projections to align monolingual word embedding spaces ( Artetxe et al . , 2016 ; Smith et al . , 2017 ; Lample et al . , 2018 , inter alia ) . Although very successful in practice , the linear nature of these methods encodes unrealistic simplifying assumptions ( e.g. all translations of a word have similar embeddings ) . In this paper , we show it is possible to produce much higher quality lexicons without these restrictions by introducing new methods that combine ( 1 ) unsupervised bitext mining and ( 2 ) unsupervised word alignment . We show that simply pipelining recent algorithms for unsupervised bitext mining ( Tran et al . , 2020 ) and unsupervised word alignment ( Sabet et al . , 2020 ) significantly improves bilingual lexicon induction ( BLI ) quality , and that further gains are possible by learning to filter the resulting lexical entries . Improving on a recent method for doing BLI via unsupervised machine translation ( Artetxe et al . , 2019 ) , we show that unsupervised mining produces better bitext for lexicon induction than translation , especially for less frequent words . These core contributions are established by systematic experiments in the class of bitext construction and alignment methods ( Figure 1 ) . Our full induction algorithm filters the lexicon found via the initial unsupervised pipeline . The filtering can be either fully unsupervised or weakly-supervised : for the former , we filter using simple heuristics and global statistics ; for the latter , we train a multi-layer perceptron ( MLP ) to predict the probability of a word pair being in the lexicon , where the features are global statistics of word alignments . In addition to BLI , our method can also be directly adapted to improve word alignment and reach competitive or better alignment accuracy than the state of the art on all investigated language pairs . We find that improved alignment in sentence representations ( Tran et al . , 2020 ) leads to better contextual word alignments using local similarity ( Sabet et al . , 2020 ) . Our final BLI approach outperforms the previous state of the art on the BUCC 2020 shared task ( Rapp et al . , 2020 ) by 14 F 1 points averaged over 12 language pairs . Manual analysis shows that most of our false positives are due to the incompleteness of the reference and that our lexicon is comparable to the reference lexicon and the output of a supervised system . Because both of our key building blocks make use of the pretrainined contextual representations from mBART ( Liu et al . , Word Alignment Statistical Feature Extraction cooccurrence(good , guten ) = 2 one-to-one align(good , guten ) = 2 many-to-one align(good , guten ) = 0 cosine_similarity(good , guten ) = 0.8 inner_product(good , guten ) = 1.8 count(good ) = 2 count(guten ) = 2 We present a direct and effective framework for BLI with unsupervised bitext mining and word alignment , which sets a new state of the art on the task . From the perspective of pretrained multilingual models ( Conneau et al . , 2019 ; Liu et al . , 2020 ; Tran et al . , 2020 , inter alia ) , our work shows that they have successfully captured information about word translation that can be extracted using similarity based alignment and refinement . Although BLI is only about word types , it strongly benefits from contextualized reasoning at the token level . umich.edu/ ˜mihalcea / wpt ( en-fr and ro-en ) ; https : //web.eecs.umich.edu/ ˜mihalcea / wpt05 ( enhi )
Existing methods to induce bilingual lexicons use linear projections to align word embeddings that are based on unrealistic simplifying assumptions.
They propose to use both unsupervised bitext mining and unsupervised word alignment methods to produce higher quality lexicons.
The proposed method achieves the state-of-the-art in the bilingual lexical induction task while keeping the interpretability of their pipeline.