Some patients in this study will receive GM-CSF along with ipilimumab. In clinical trials, GM-CSF has been safely given to prostate cancer patients in combination with ipilimumab. GM-CSF is not approved by the FDA for use as treatment for prostate cancer. Studies in patients with prostate cancer suggest that GM-CSF may activate the immune system. Since ipilimumab can help keep the immune system from turning off and allow an immune reaction to occur, and GM-CSF can increase the activity of the immune system, it is possible that they may work together to increase the immune response to cancer. The use of ipilimumab in combination with GM-CSF in patients with CRPC has not been approved by the FDA.