1. An image decoding method using residue sampling comprising: when in an original image formed with at least two image components, the difference between the original image and a predicted image is referred to as a residue, generating downsampled data of residues by performing at least entropy decoding from a bitstream; upsampling the downsampled data of the residues; performing inverse residue transform of the upsampled data by transforming the upsampled data using a relation between residues of respective image components; and restoring the original image by adding the predicted image to the upsampled residue of each component wherein at least one of the generating of the downsampled data of residues, upsampling of the downsampled data of the residues, performing of the inverse residue transform, and restoring of the original image is implemented by at least one processing device.