The real cause of low libido, thickening waistlines in middle-aged men MANY middle-aged men find their waistlines expanding and their sex drive shrinking, symptoms a new study suggests can be traced to a hormone deficiency - but not the one you might imagine. Previously, a drop in testosterone production might have been suspected as the culprit, but researchers said a decline in estrogen may be part of the problem.