1. A disparity estimation apparatus comprising: an energy calculator to calculate an energy related to stereo matching of each of a left image and a right image which constitute a stereo image; a map generator to generate a visibility map for determining an error in disparity between the left image and the right image using the calculated energy; an energy recalculator to recalculate the energy with respect to a region including a visibility error generated due to the error in disparity in the visibility map; and a disparity determiner to determine disparity from the stereo image using the recalculated energy of each of the left image and the right image.