1. An image stabilizing apparatus comprising: at least one hardware processor which implements: an image adjusting unit which: sets a characterizing point checking region in a currently input image frame, compares the currently input image frame with a reference image to determine whether the currently input image frame is shaken, based on the reference image and the characterizing point checking region, and adjusts the currently input image frame by a shaking amount if it is determined that the current image frame is shaken; a reference image setting unit which extracts an image frame which is a least shaken among a plurality of image frames included in image data taken for a certain period of time, and sets the extracted image frame as the reference image; and a characterizing point checking region setting unit which sets the characterizing point checking region by using a plurality of characterizing points in the plurality of image frames, wherein the least shaken image frame is set as the reference image by using a distance between a center point and the plurality of characterizing points in each of the plurality of image frames.