1. An image processing apparatus comprising: a block division unit that divides at least one image frame out of a plurality of image frames constituting moving image information into a plurality of blocks; a motion vector calculation unit that tracks each of the blocks that are obtained by the block division unit, in one of two image frames before and after the one image frame, and calculating a plurality of motion vectors of the blocks, respectively; a global motion vector calculation unit that compares one of the motion vectors calculated by the motion vector calculation unit with each of other motion vectors, the one motion vector serving as an evaluation target, thereby calculating a global motion vector of the one image frame; and a block identification unit that excludes blocks present within a predetermined distance from one block related to one motion vector serving as the evaluation target from comparison targets the motion vectors of which are to be compared with the one motion vector, when the global motion vector calculation unit calculates the global motion vector.