1. An image processing apparatus comprising: position estimating means for estimating the position of a second point representing a tracking point in an image of a temporally next unit of processing, the second point corresponding to a first point representing the tracking point in an image of a temporally previous unit of processing; generating means for generating estimated points serving as candidates of the first point when the position of the second point is inestimable; determining means for determining the second point in the next unit of processing on the basis of the estimation result of the position estimating means when the position of the second point in the next unit of processing is estimable; and selecting means for selecting the first point from among the estimated points when the position of the second point is inestimable, wherein the determining means includes: evaluation value computing means for computing an evaluation value representing a correlation between pixels of interest representing at least one pixel including the first point in the temporally previous unit of processing and the corresponding pixels representing at least one pixel in the temporally next unit of processing and defined on the basis of a motion vector of the pixels of interest; variable value computing means for computing a variable value representing the variation of a pixel value with respect to the pixels of interest; and accuracy computing means for computing the accuracy of the motion vector, and the variable value computing means computes the variable value representing the sum of values obtained by dividing the sum of absolute differences between the pixels of interest and the adjacent pixels that are adjacent to the pixels of interest in a block including the pixels of interest by the number of the adjacent pixels.