1. An apparatus for synthesizing three-dimensional (3D) images, comprising: an image input/output unit configured to receive photographed images of a reference pattern formed on a ground and a non-ground plane from a plurality of wide angle cameras mounted on a vehicle, and to output the photographed images; an image arrangement estimation unit configured to extract feature points from the reference patterns of the image input from the image input/output unit, and to estimate a relative location and installation angle of each of the cameras using known physical location information of the extracted feature points; a sphere mapping unit configured to obtain optical parameters for correction of lens distortion including an optical center of a lens using the photographed images, and to map each of the images to a surface of a Gaussian sphere using obtained optical center; a virtual space arrangement unit configured to, using the relative location and installation angle of the camera estimated by the image arrangement estimation unit, change an angle and distance of the Gaussian sphere so that the Gaussian sphere coincides with the real reference patterns, and arrange the Gaussian spheres in a 3D virtual space; and a single image acquisition unit configured to obtain a single 3D image by mapping the images arranged in the 3D virtual space onto an inner surface of a 3D sphere corresponding to a single large sphere.