Ordering the removal of the flag is tantamount to thought control, this owner has no obligation to be sensitive to everyone's dislikes and what their subjective thought might find offensive, and, you don't know why he has the flag on his wall, hankering after a past which might include fond memories for the owner does not necessarily mean he supports a crime against humanity or racism.