1. A method for controlling a video segmentation apparatus, comprising: receiving an image corresponding to a frame of a video; estimating a motion of an object to be extracted from the received image; determining a plurality of positions of windows corresponding to the object; adjusting at least one of a size and a spacing of at least one window located at a position of the plurality of determined positions of the windows based on an image characteristic; and extracting the object from the received image, based on the at least one window of which the at least one of the size and the spacing is adjusted.