1 : An image processing apparatus comprising: position estimating means for estimating the position of a second point representing a tracking point in an image of a temporally next unit of processing, the second point corresponding to a first point representing the tracking point in an image of a temporally previous unit of processing; generating means for generating estimated points serving as candidates of the first point when the position of the second point is inestimable; determining means for determining the second point in the next unit of processing on the basis of the estimation result of the position estimating means when the position of the second point in the next unit of processing is estimable; and selecting means for selecting the first point from among the estimated points when the position of the second point is inestimable.