The end-to-end nature of neural machine translation (NMT) removes many ways of manually guiding the translation process that were available in older paradigms. Recent work, however, has introduced a new capability: lexically constrained or guided decoding, a modification to beam search that forces the inclusion of pre-specified words and phrases in the output. However, while theoretically sound, existing approaches have computational complexities that are either linear (Hokamp and Liu, 2017) or exponential (, 2017) in the number of constraints. We present a algorithm for lexically constrained decoding with a complexity of O(1) in the number of constraints. We demonstrate the algorithms remarkable ability to properly place these constraints, and use it to explore the shaky relationship between model and BLEU scores. Our implementation is available as part of Sockeye.
was novel in that it allowed the specification of arbitrary target-side words as hard constraints, implemented entirely as a restructuring of beam search. A related approach was that of , who similarly extended beam search with a finite state machine whose states marked completed subsets of the set of constraints, at an exponential cost in the number of constraints. Both of the approaches make use of target-side constraints without any reference to the source sentence. A related line of work is to attempt to force the use of particular translations of source words and phrases, which allows the incorporation of external resources such as glossaries or translation memories. propose a variant on beam search that interleaves decoder steps with attention-guided checks to ensure that glossary annotations on the source sentence are generated by the decoder. Their implementation also permits discontinuous phrases and translation options (these capabilities are not available in , but could be added). A softer approach treats external resources as suggestions that are allowed to influence, but not necessarily determine, the choice of words @cite_2 and also phrases .
@cite_2
2410217169
Neural machine translation (NMT) often makes mistakes in translating low-frequency content words that are essential to understanding the meaning of the sentence. We propose a method to alleviate this problem by augmenting NMT systems with discrete translation lexicons that efficiently encode translations of these low-frequency words. We describe a method to calculate the lexicon probability of the next word in the translation candidate by using the attention vector of the NMT model to select which source word lexical probabilities the model should focus on. We test two methods to combine this probability with the standard NMT probability: (1) using it as a bias, and (2) linear interpolation. Experiments on two corpora show an improvement of 2.0-2.3 BLEU and 0.13-0.44 NIST score, and faster convergence time.