1. An apparatus for processing an image to generate a second image of higher spatial resolution from a first image of lower spatial resolution which is composed of image data made up of a plurality of component signals, comprising: acquiring means for acquiring first pixel data of the first image; and predicting means for predicting a first component signal of second pixel data of the second image using at least first and second component signals out of component signals of said first pixel data, and predicting a second component signal of the second pixel data of said second image using at least said first and second component signals out of the component signals of the first pixel data.