1. An image processing apparatus for detecting a motion vector between a reference image and a comparison image, comprising: an image input for inputting the reference image and the comparison image; a receiver for receiving a user's operations; a reference-area specifier for specifying a first reference area in the reference image according to the user's operation received by the receiver; a divider for setting a boundary used for dividing the first reference area into a foreground and a background, according to the user's operation received by the receiver; a comparison-area specifier for specifying a plurality of comparison areas in the comparison image; a first calculator for calculating a first degree of appropriateness indicating a degree of correlation between the foreground of the first reference area and each of the plurality of comparison areas; a generator for generating a second reference area by differentiating a portion close to the boundary of the first reference area in a direction perpendicular to the boundary; a second calculator for calculating a second degree of appropriateness indicating a degree of correlation between a portion close to the boundary of the second reference area and each of the plurality of comparison areas; a determination part for determining a matching area corresponding to the first reference area among the plurality of comparison areas according to a sum of the first and second degrees of appropriateness; and an operator for calculating the motion vector between the reference image and the comparison image according to a position of the first reference area in the reference image and a position of the matching area in the comparison image.