1. An apparatus for displaying a holographic three-dimensional (3D) image, the apparatus comprising: an image segmenter configured to segment an original image into a plurality of segments, and adjust a number of the segments per unit volume or the number of the segments per unit area; a calculator configured to calculate diffraction fringe pattern information for displaying each of the plurality of segments as a 3D holographic image; and a light modulator configured to form a diffraction fringe pattern according to the diffraction fringe pattern information and generate the 3D holographic image corresponding to the original image by modulating a light incident on the diffraction fringe pattern.