1. An image processing apparatus comprising: a first cost calculating section configured to calculate a degree of similarity between a first pixel in a first image and each of a plurality of second pixels located in a horizontal-direction first range in a second image, extract a second pixel with a highest similarity degree, and calculate a disparity from the first pixel; a second cost calculating section configured to calculate, only for the extracted second pixel, a degree of similarity between a plurality of third pixels existing in a vertical-direction second range that includes the extracted second pixel and the first pixel, in the second image, and extract a third pixel with a highest similarity degree; an amount-of-difference buffer configured to record a difference between a vertical-direction position of the extracted third pixel and the first pixel as an amount of difference of the extracted third pixel; and a search position setting section configured to determine positions of the second pixels for which a degree of similarity to the first pixel for which a disparity is calculated next is calculated, according to data of the amount-of-difference buffer.