First, we develop a new discriminative term-weighting metric called TF-KLD, which is applied to the term-context matrix before factorization. On a standard paraphrase identification task (Dolan et al., 2004) , this method improves on both traditional TF-IDF and Weighted Textual Matrix Factorization (WTMF;<cite> Guo and Diab, 2012)</cite> .