1. An image processing device for generating a disparity image from a first image and a second image, comprising: a vertical edge extraction unit configured to calculate a vertical edge component of each pixel in the first image; a horizontal segmentation unit configured to divide a horizontal line in the first image into a plurality of segments using, as a border between each of the segments, a pixel for which the vertical edge component has a magnitude exceeding a predetermined threshold; a disparity calculation unit configured to calculate a disparity between each of the segments and a region of the second image similar to the segment; and a disparity image generation unit configured to generate the disparity image in accordance with the disparity calculated for each of the segments.