1. An image processing apparatus that generates composite image data by combining multi-viewpoint image data corresponding to a plurality of viewpoints, the device comprising: a first input unit configured to input a virtual focus distance of the composite image data; a second input unit configured to input distance information indicative of a distance of a subject in the composite image data from a viewpoint included in the plurality of viewpoints; a determination unit configured to input information indicative of a positional relationship of the plurality of viewpoints and determine, based on the virtual focus distance, the distance information and the positional relationship, a size of a blurring filter, to be applied to the multi-viewpoint image data, which blurs an image; a filter processing unit configured to perform filter processing on the multi-viewpoint image data using the blurring filter of the size determined by the determination unit; and a generation unit configured to generate composite image data by combining the multi-viewpoint image data on which the filter processing has been performed.