Prostate cancer begins when cells in the prostate gland start to grow uncontrollably. The prostate is a gland found only in males. It makes some of the fluid that is part of semen. The prostate is below the bladder and in front of the rectum. The size of the prostate changes with age. In younger men, it is about the size of a walnut, but it can be much larger in older men. Just behind the prostate are glands called seminal vesicles that make most of the fluid for semen.