In this paper, we focus on improving a state-ofthe-art counting model, LexVec <cite>(Salle et al., 2016)</cite> , which performs factorization of the positive pointwise mutual information (PPMI) matrix using window sampling and negative sampling (WSNS).