We conducted two series of experiments to validate the efficiency of our approach and the quality of the models we learned using it: An intrinsic study of language model perplexity using the standard One Billion Word benchmark<cite> (Chelba et al., 2013)</cite> and an extrinsic end-to-end statistical machine translation task that uses an LSTM as one of several feature functions in re-ranking.