1. An image generating apparatus comprising: an image acquirer which acquires a photo shooting image formed by a plurality of sub images shot from each of a plurality of viewpoints; an extractor which extracts, from sub-pixels forming the sub images, sub-pixels corresponding to an object whose composition is to be modified as pixels to be modified; a modifier which modifies a correspondence relationship with sub-pixels extracted as the pixels to be modified among correspondence relationships between the sub-pixels and pixels to be reconstructed forming a reconstructed image which is defined on a predetermined reconstruction plane; a generator which generates a reconstructed image in which the composition of the object has been modified by calculating pixel values of the pixels to be reconstructed from pixel values of the sub-pixels using the modified correspondence relationship; and a depth coefficient acquirer which acquires, for each of the sub-pixels, a depth coefficient indicating a depth of an object corresponding to the sub-pixel; wherein the generator, referring to a correspondence relationship modified by the modifier, compares depth coefficients of the pixels to be modified and depth coefficients of the sub-pixels which are not the pixels to be modified, and acquires, giving priority to, the pixel values having less deep depth coefficients in acquiring pixel values of pixels to be reconstructed corresponding to the pixels to be modified.