1. An image processing apparatus that calculates information according to an interframe motion of a moving image based on motion vectors detected in the moving image, the image processing apparatus comprising: at least one processor and at least one memory cooperating to act as units comprising: (1) a detection unit configured to detect motion vectors in a frame of a moving image; (2) a specifying unit configured to specify, from the motion vectors detected by the detection unit, one or more motion vectors that belong to a region of interest of a plurality of regions obtained by dividing the frame; (3) a determination unit configured to, in a case where a plurality of motion vectors that belong to the region of interest are specified by the specifying unit, determine, from the plurality of motion vectors that belong to the region of interest, a representative motion vector of the region of interest; (4) an extraction unit configured to, from the plurality of motion vectors that belong to the region of interest other than the representative motion vector of the motion vectors that belong to the region of interest extract, as motion vectors having components that are similar to the representative motion vector of the region of interest, only motion vectors that satisfy a predetermined condition; and (5) a calculation unit configured to, where the specifying, determining, and extracting are performed to a plurality of regions in the frame, calculate a rotation matrix as the information according to the interframe motion of the moving image based on one or more motion vectors of the extracted motion vectors of in each of the plurality of regions, wherein the calculation unit selects a predetermined number of motion vectors from motion vectors extracted in each of the plurality of regions, and calculates a rotation matrix based on the selected motion vectors, wherein the predetermined number is equal to or more than 4, and wherein the calculation unit (a) compares (1) the difference between a terminal point of a motion vector extracted in each of the plurality of regions and a point to which a motion vector extracted in each of the plurality of regions is projected using a calculated rotation matrix with (2) a first tolerance, (b) counts a number of motion vectors that do not exceed the first tolerance, and (c) determines the rotation matrix by changing a combination of the motion vectors to be selected from motion vectors extracted in each of the plurality of regions, until a counted value satisfies a predetermined value.