All patients will undergo a rectal biopsy before treatment begins and during treatment. Tissue will be submitted to H. Lee Moffitt Cancer Center for tumor marker assays. Tissue from the excised tumor at the time of surgery will also be sent for the same studies. Between day 10 and 14 of treatment, colonoscopy/sigmoidoscopy is mandatory to ensure having a quantitative estimate of the tumor shrinkage and to obtain a repeat rectal biopsy. All patients will undergo radiation therapy concurrently with oral topotecan. Patients will receive 180 cGY per fraction to a total dose of 4500cGy(conventional fractionation) to the pelvis using a 3 or 4 filed technique and high energy photons. Standardized fields for rectal cancer will be used to include the true pelvis (mid sacrum to at least 2-3 centimeters below the inferior aspect of the tumor volume).