but , as the chair had pointed out earlier , if people knew you were a wizard because you were wearing a pointy hat , then if you took the pointy hat off , they 'd think you were just some rich merchant or something .