Trump is a millionaire celebrity as well. His personal views  are no more or no less significant than any one else.  However, when Trump speaks as the president he has a responsibility to all Americans not just a few.   This issue is no different than  the flag burners whose rights were held up by SCOTUS or the Courts decisions that one does not have too recite the Pledge of Allegiance.  
Likewise one group does not have more rights than another.  Courts have held the rights of far rights group to assemble and march equally with the rights of a  church group to hold similar marches. 

Nothing in the Constitution says you have to like how individuals chose to use their rights.