1. An apparatus to code an image, the apparatus comprising: a low resolution image coding unit that codes a plurality of low resolution images respectively corresponding to a plurality of input images using motion estimation and compensation; and a residual image coding unit that performs intra mode coding with respect to a plurality of residual images, the residual images relating to differences between reconstructed images of the coded low resolution images and the input images, the residual image coding unit performing the intra mode coding by considering a temporal correlation between neighboring ones of the residual images.