1. An X-ray therapy system comprising: a ring-shaped gantry in which a patient is to be arranged; a positron emission tomography (PET) detection device in the gantry and configured to perform PET on the patient; an irradiation unit in the gantry and configured to radiate X-rays to the patient; and a rotation-driving unit for causing the gantry to perform a rotational movement around a predetermined rotation center, wherein the gantry includes a concave portion on an inner circumferential surface thereof, wherein the concave portion includes a bottom surface at an outer peripheral side of the gantry and an opening at an inner peripheral side of the concave portion, wherein the gantry further includes a cover member for covering the opening of the concave portion, the cover member being configured to block the X-rays or scattered rays due to the X-rays from the irradiation unit, wherein the PET detection device includes: photon detection units to detect photons emitted from the patient based on nuclear decay of a tracer nuclide administered to the patient; and a photon detection unit-moving device configured to move one of the photon detection units with respect to the gantry, wherein the photon detection units are on opposite sides from each other with the patient therebetween, wherein a first end of the photon detection unit-moving device is fixed to the one of the photon detection units, and a second end of the photon detection unit-moving device is fixed to the bottom surface of the concave portion, wherein the rotation-driving unit causes the irradiation unit and the PET detection device to perform the rotational movement around the predetermined rotation center by causing the gantry to perform the rotational movement around the predetermined rotation center, wherein, when detecting photons in the PET, the photon detection unit-moving device is configured to arrange the one of the photon detection units at a position separated from the patient by a predetermined distance, and wherein, when the X-rays are radiated from the irradiation unit to the patient, the one of the photon detection units moves away from the patient and is housed in a retraction position inside the concave portion before radiation of the X-rays starts.