1. An image processing device which processes an input image to generate a processed image, comprising: smoothing means for extracting a smoothing tap and smoothing a target processing image that is the input image on the basis of pixel values within the smoothing tap, the smoothing tap being variable in size and formed by a plurality of pixels centered on each target pixel of the target processing image; class tap extracting means for extracting a class tap formed by a plurality of pixels centered on each target pixel in the smoothed target processing image; class code determining means for generating a code corresponding to a characteristic of variation of pixel values within the class tap, and determining a class code, which includes a size of the smoothing tap and the code corresponding to a characteristic of variation of pixel values; and pixel value computing means for reading tap coefficients corresponding to the determined class code from a storing section, and multiplying individual pixel values forming a prediction tap extracted from the smoothed target processing image, by the tap coefficients to calculate pixel values of the processed image.