1. An image processing apparatus, comprising: an image input portion for receiving a base image and a reference image; a first window setting portion for setting a first window on one of said base image and said reference image; a second window setting portion for setting a second window on the other of said base image and said reference image; an extraction portion for extracting at least one image feature value from the image in said first window; a modification portion for modifying said first window in at least one of a size and a shape thereof based on extracted said image feature value and modifying said second window in a size or a shape thereof in accordance with modification to said first window; and an operation portion for performing an operation to specify correspondence between the image in the modified first window and the image in the modified second window, wherein said extraction portion measures an edge value in a central portion of said first window, and said modification portion reduces the size of said first window when the edge value in the central portion of said first window is equal to or greater than a prescribed value.