How to Train good Word Embeddings for Biomedical NLP
The quality of word embeddings depends on the input corpora, model architectures, and hyper parameter settings. Using the state of the art neural embedding tool word2vec and both intrinsic and extrinsic evaluations, we present a comprehensive study of how the quality of embeddings changes according to these features. Apart from identifying the most influential hyper parameters, we also observe one that creates contradictory results between intrinsic and extrinsic evaluations. Furthermore, we find that bigger corpora do not necessarily produce better biomedical domain word embeddings. We make our evaluation tools and resources as well as the created state of the art word embeddings available under open licenses from https:/github.com/ cambridgeltl/BioNLP 2016.
BioNLP@ACL