Word Embedding Calculus in Meaningful Ultradense Subspaces
We decompose a standard embedding space into interpretable orthogonal subspaces and a "remainder" subspace. We consider four interpretable subspaces in this paper: polarity, concreteness, frequency and part of speech (POS) subspaces. We introduce a new calculus for subspaces that supports operations like " 1 x hate love" and "give me a neutral word for greasy" (i.e. oleaginous) This calculus extends analogy computations like "king man+woman queen" For the tasks of Antonym Classification and POS Tagging our method outperforms the state of the art. We create test sets for Morphological Analogies and for the new task of Polarity Spectrum Creation.
ACL