It's typically restricted because hats can be used to identify gang members, or hide faces. Not saying they're good reasons, but they're reasons.

Edit: great point, hats are also restricted because it can be considered disrespectful.

Edit2: I guess also cheating... none of these seem like really good reasons for no hats at any time...