1. A virtual viewpoint image generation method comprising: a step of obtaining plural images of an object taken by plural cameras; a step of determining a virtual viewpoint that is a position from which the object is viewed; and a step of generating a virtual viewpoint image that is an image of the object viewed from the virtual viewpoint based on the obtained images of the object, the step of generating the virtual viewpoint image comprising: a step 1 of setting projection planes having a multi-layered structure; a step 2 of obtaining each corresponding point, on the images of the object, corresponding to a projection point of a projection plane; a step 3 of determining color information or brightness information of each projection point based on color information or brightness information of corresponding points; a step of calculating, for each of the projection points overlapping when viewed from a reference viewpoint in a space, a degree of probability that the object exists at a distance corresponding to a position of the projection point based on a degree of correlation of the corresponding points or neighborhoods of the corresponding points; a step 5 of performing mixing processing on color information or brightness information of reference points overlapping when viewed from the virtual viewpoint according to the degree of probability of existence of the object so as to determine color information or brightness information of each pixel of the virtual viewpoint image; and a step 6 of repeating steps from the step 1 to the steps 5 for every point corresponding to pixels of the virtual viewpoint image.