However, these previous methods all suffer from one or more of three short-comings. <cite>Klementiev et al. (2012)</cite> ; ; Gouws et al. (2014) all learn their representations using a word-level monolingual objective. This effectively means that compositionality is not encouraged by the monolingual objective, which may be problematic when composing word representations for a phrase or document-level task.