Men need to make with women while they are young and their reproductive organs still work. 
A better explanation is that males' reproductive fate depends more strongly than females' on competing when they are young.