1. An image processing apparatus that generates a joined image by joining a plurality of textures based on a plurality of images, the image processing apparatus comprising: a derivation unit configured to derive a motion between images of the plurality of images; a frame memory in which a joined image frame is set and the plurality of textures that form the joined image are written; a pixel structure determination unit configured to determine the joined image frame as a first rectangular pixel structure when an user designates image pickup direction information as a horizontal direction, and configured to determine the pixel structure determination unit determines the joined image frame as a second rectangular pixel structure when the user designates the image pickup direction as a perpendicular direction, a texture writing unit configured to write the plurality of textures into the frame memory based on the motion between the images, wherein the texture writing unit writes a first texture among the plurality of textures into an area that is not in contact with an edge of the joined image frame, wherein, in the first rectangular pixel structure, the number of the pixels in the horizontal direction is larger than the number of the pixels in the perpendicular direction, wherein in the second rectangular pixel structure, the number of the pixels in the perpendicular direction is larger than the number of the pixels in the horizontal direction, and wherein a total number of the pixels in the first rectangular pixel structure is the same as a total number of the pixels in the second rectangular pixel structure, wherein the derivation unit, the pixel structure determination unit and texture writing unit includes at least one of a CPU, a memory and an LSI.