We evaluate our approach on various tasks that require contextual understanding of words, combining existing and new test datasets and evaluation metrics: word-sense induction ( (Koeling et al., 2005; Bartunov et al., 2015) ), contextual word similarity<cite> ((Huang et al., 2012</cite> ) and a new test set), and relevance detection ( (Arora et al., 2016) and a new test set).