1. An image processing device comprising: a photographed image acquisition section for sequentially acquiring photographed images of a relatively moving object, the photographed images changing gradually; a characteristic extraction section for extracting characteristic points from a photographed image acquired with the photographed image acquisition section; a characteristic point tracking section for searching a plurality of the photographed images acquired before or after the photographed image from which the characteristic points have been extracted, for candidate corresponding points corresponding to the characteristic points, to keep track of the characteristic points; and a processing section for determining corresponding points from the candidate corresponding points based on tracking results to calculate coordinates of a photographing position or the object using the determined corresponding points, wherein the tracking results include a screen relative movement amount obtained through a projective transformation using two images out of the sequentially acquired photographed images, or movement vectors of characteristic points between the two photographed images, or movement tracks obtained by sequentially linking the screen relative movement amount or the movement vectors, wherein the characteristic point tracking section uses the photographed image from which the characteristic points have been extracted and a plurality of the photographed images sequentially acquired prior or subsequent thereto to search for candidate corresponding points corresponding to the characteristic points; forms the movement vectors based on positional relationship between candidate corresponding points found and the characteristic points; selects candidate corresponding points judged to probably be corresponding points corresponding to the characteristic points based on the movement vectors and the screen relative movement amount; searches a plurality of the photographed images sequentially acquired prior or subsequent thereto, for candidate corresponding points corresponding to the characteristic points and the corresponding points; and sequentially repeats forming movement vectors, selecting candidate corresponding points and searching for candidate corresponding points.