What exactly is a property? How is a property $P$ formally defined in mathematics?
I mean for example if $f$ is a morphism from an object $X$ to $Y$ in some category, then somehow I feel that "has codomain $Y$" is too broad to be considered a property...

The wiki says:

In mathematical terminology, a property p defined for all elements of a set X is usually defined as a function p: X → {true, false}, that is true whenever the property holds; or equivalently, as the subset of X for which p holds; i.e. the set {x| p(x) = true}; p is its indicator function.

This is plausible, but strictly speaking one would like to allow X to be a proper class so that one could talk about a property of groups, topologies, etc.
Really, the natural language meaning is enough to do most mathematics. 
