1. An image processing apparatus comprising: setting means for setting a composition pattern corresponding to an input image, on the basis of the number of salient regions to which attention is directed in the input image, and a scene of the input image; determining means for determining a crop region in the input image which is optimal for an image to be cropped in the composition pattern from the input image, on the basis of the composition pattern set by the setting means; motion direction determining means for determining a direction of motion of a subject to which attention is directed; overall motion calculating means for finding overall motion of the input image; and local motion calculating means for finding motion of each of the salient regions, wherein the motion direction determining means determines a direction of motion of the subject on the basis of an orientation of the overall motion of the input image found by the overall motion calculating means, and an orientation of the motion of each of the salient regions found by the local motion calculating means.