People resent that other people have privilege and for a sort of aristocracy.
What's more, in our egalitarian times people resent the privilege that serves to insulate a somewhat random quasi-aristocracy from contact with citizens.