If a human learns the fact "Valentina Tereshkova was the first woman to travel to space", they can also correctly answer "Who was the first woman to travel to space?". This is such a basic form of generalization that it seems trivial. Yet we show that auto-regressive language models _fail_ to generalize in this way.