I am not justifying this guy's actions but he did not violate anyone's rights. 
As a number of women find catcalling offensive, that alone does not make it a rights violation...It's been tested in court a number of times. 
For all we know his comment was more directed towards the life jacket than the female. Even if he was directed at the woman, it was inappropriate and poor taste but that doesn't justify ousting him in my opinion. He made a defensive statement afterwards, if had continued to be unruly or inappropriate to any gender then yes he should of been ousted for being a disruption. 

Again, i am not condoning his actions just feel your inclusion of "rights violation" is based on feeling and not fact.