1. An apparatus for generating a hologram pattern, the apparatus comprising: a processor configured to quantize depth layers of a depth image based on depth image information of a three-dimensional (3D) object, identify a quantized depth layer having a number of points of 3D object in the quantized depth layer less than or equal to a threshold value among the quantized depth layers, merge the identified quantized depth layer having the number of points of the 3D object less than or equal to the threshold value with an adjacent depth layer, and generate a hologram pattern corresponding to the 3D object in a hologram plane using color image information of the 3D object and a point of the 3D object included in the quantized depth layers.