1. A method of processing video signals representing multiple images, comprising: identifying pixels in an image that should not be included in performing motion compensation of images in the multiple images; using a hole-filling process to replace the identified pixels with values generated from values in a background image to form a hole-filled image; performing motion estimation on the hole-filled image to generate a motion vector field; using the motion vector field to perform motion-compensated processing on the hole-filled image; and combining a result of the motion-compensated processing with the identified pixel to generate an output image.