1. An image photographing apparatus comprising: a first photographing unit configured to photograph an image by using a phase difference image sensor; a second photographing unit configured to be spaced on a side of the first photographing unit; a controller configured to generate a first depth map by using a first image photographed by the first photographing unit and generate a second depth map by using a second image photographed by the second photographing unit; and an image processor configured to generate a three-dimensional (3D) image by using the first depth map and the second depth map.