Noel was brought up at Trentholme, the home of his maternal grandfather, located 0.8 miles north of Goddards, and opposite the Elmbank Hotel which, at the time, was the home of Sidney Leetham, an uncle of Kathleen Leetham (1892–1980).