Hadrian thought about it.  "Aunt Petunia complained that I was always trying to head North from the first time I toddled.  Uncle Vernon complained a lot that no matter how much he shook me up, I would always face North given enough time."