Neural Language Models can be tracked back to [3] , and more recently deep bi-directional language models (biLM) <cite>[33]</cite> have successfully been applied to word embeddings in order to incorporate contextual information.