Large Language Models (LLMs) trained on internet-scale data perform extremely well on tasks relating to reasoning, common-sense, and world-knowledge. In particular, the range of knowledge captured by LLMs like GPT-4 (OpenAI, 2023) and Llama-2 (Touvron et al., 2023b) is significantly wider than that of an average person. However, recent research (Berglund et al., 2023; Allen-Zhu & Li, 2023a;b) uncovered a curious flaw in the knowledge capabilities of LLMs, coined the _reversal curse_. They experimentally showed that even the currently most powerful LLMs are incapable of "reversing" facts they had learned. For example, standard LLMs cannot correctly answer "What's the capital of France?" even if the training data contains "Paris is the capital of France", unless it also contains text where "France" is followed by "Paris", such as "As the capital of France, Paris has...".