1. A method of estimating a disparity of a stereo image, comprising: obtaining a plurality of images having different resolution levels for the stereo image; estimating a disparity map for a lowest-resolution level image; estimating a coarse disparity map for an upper resolution level image by using the disparity map; obtaining a fine disparity map for the upper resolution level image by using the coarse disparity map; and outputting the fine disparity map as a final disparity map for the stereo image if the upper resolution level image has a resolution of a highest level, wherein the estimating of the disparity map for the lowest-resolution level image includes obtaining a disparity map which minimizes energy including a smooth energy, the smooth energy being based upon a difference in disparities between neighboring pixels in the disparity map.