What would explain the Reversal Curse in auto-regressive LLMs? We mostly leave this for future work. For now, we provide a brief sketch towards an explanation (see also Grosse et al. (2023)). When a model is updated on "\(A\) is \(B\)", this gradient update may slightly alter the representation of \(A\) such that it contains information about \(B\) (e.g. in the middle MLP layers as per Geva et al. (2022; 2023)). It would make rational sense for this gradient update to also alter the representation of \(B\) to contain information about \(A\).