I agree, it might be a good idea to have quiet word with your supervisor or HR manager about this. Even if you don't want to make a formal complaint, they can still make a note of what has been said...