1. An image pickup apparatus, comprising: an imaging portion which photoelectrically converts a subject image and obtains a frame image of a subject; a first motion vector deciding portion which decides a motion vector Vij of each of segmented regions Dij (i=1 to M, j=1 to N) made by segmenting the frame image of the subject obtained by the imaging portion into multiple matrix domains of M lines×N rows; an AF evaluation value calculating portion which calculates an AF evaluation value indicating a degree of sharpness of the subject image from the frame image of the subject obtained by the imaging portion; a coefficient deciding portion which decides a weighting coefficient αij of each motion vector Vij based on the AF evaluation value calculated by the AF evaluation value calculating portion; and a second motion vector deciding portion which decides an entire motion vector V of the frame image of the subject by weighting the motion vector Vij based on the weighting coefficient αij.