1. A method of processing a holographic image, the method comprising: generating a first complex image corresponding to a first resolution by performing frequency transformation on input image data including color data and depth data of each of pixels in a left-eye two-dimensional (2D) image and a right-eye 2D image having the first resolution, the first complex image comprising real component image data of real values corresponding to each pixel in the left-eye (2D) image and the right-eye 2D image of the input image data and imaginary component image data or imaginary values corresponding to each pixel in the left-eye 2D image and the right-eye 2D image of the input image data; up-scaling the first complex image having the first resolution to a second complex image having a second resolution, wherein the first resolution is less than the second resolution; encoding complex values corresponding respectively to pixels of the second resolution in the second complex image that has been up-scaled, into integer values; and generating a computer generated hologram (CGH) image corresponding to each of a left-eye image and a right-eye image having the second resolution by using the encoded integer values; displaying the computer generated hologram image corresponding to the left-eye image and the computer generated hologram image corresponding to the right-eye image.