Every building had walls. While on the Horcrux hunt, Harry had explained to Hermione that according to Dumbledore and the memories he had waded through with the older wizard, Voldemort was a sentimental idiot.