1. An apparatus for generating a panoramic image, comprising: a mapping unit to project a previously captured image and a currently captured image onto a predetermined curved surface so as to form a first mapping image and a second mapping image, respectively; a calculation unit, executed by at least one processor, to calculate movement values in predetermined directions for maximizing a color similarity between overlapped sections of the first and second mapping images, by using candidate movement value of each of the predetermined directions; and an image aligning unit to generate a panoramic image using the first and second mapping images and the calculated movement values, wherein the predetermined directions include at least one of pitching direction, a panning direction, a rolling direction and a vertical displacement, wherein the calculation unit calculates the movement values after projecting one of the first and the second mapping image onto a plane including the other without considering a focal length of the apparatus, and wherein the candidate movement value is selected by the calculation unit among preset angles in each of the predetermined directions.