1. An apparatus for scaling image data, comprising: an interpolation filter configured to upscale a source image to an intermediate image by a scale factor of 2 N , where N>1; an average filter configured to downscale the intermediate image to a destination image, wherein the average filter is configured to average the intermediate image on a two-by-two pixel basis, the average filter comprising 2 N-1 horizontal average filters; and a line buffer configured to store output from the average filter, wherein the line buffer size is equal to half of a length of the destination image plus half of a length of a preceding line in the intermediate image.