RetroGAN is a system that builds on [1]} by utilizing a CycleGAN-like architecture (i.e., we use a pair of GANs cyclicly but our layers are different from the original CycleGAN). We chose the CycleGAN-like architecture because, in our domain, the cycle-consistency constraints can enforce a one-to-one mapping from original embeddings to retrofitted embeddings. This mapping guarantees that unseen concepts will have their own, unique retrofitted counterparts. We use RetroGAN to learn the mapping of Attract-Repel [2]} retrofitting (with the synonymy and antonymy constraints from the Attract-Repel paper[2]}) on a subset of static word embeddings (i.e.., FastText [4]}, and Numberbatch [5]}), and perform post-specialization on the entire set.
