1. A method of receiving video data, comprising: receiving two or more original images at a processing device; estimating a first image of higher resolution than the original image from the original image; generating a simulated image of the original image from the first higher resolution image by applying a low pass filter to the higher resolution image, producing a low pass filtered image, and shifting the low pass filtered image according to a sub-pixel motion vector to produce the simulated image; deriving difference data between the original image and the simulated image; upscaling the difference data, wherein upscaling includes weighting the difference data with an adjusted confidence value, wherein the confidence value is adjusted based upon local information; and using the upscaled difference data and the first higher resolution image to estimate a second higher resolution image to replace the first higher resolution image.