Exponentially Decaying Bag of Words Input Features for Feed Forward Neural Network in Statistical Machine Translation
Recently, neural network models have achieved consistent improvements in statistical machine translation. However, most networks only use one hot encoded input vectors of words as their input. In this work, we investigated the exponentially decaying bag of words input features for feed forward neural network translation models and proposed to train the decay rates along with other weight parameters. This novel bag of words model improved our phrase based state of the art system, which already includes a neural network translation model, by up to 0.5% BLEU and 0.6% TER on three different translation tasks and even achieved a similar performance to the bidirectional LSTM translation model.
ACL