In this section, we describe three different neural networks to pre-train word representations: the model implemented by <cite>Alikaniotis et al. (2016)</cite> and the two error-oriented models we propose in this work.