1. An image signal processing device that processes an image signal constituting a subject image captured by an image-capturing element, comprising: an average calculating unit that calculates pixel output averages for individual colors in each of a plurality of partition areas into which a light-receiving surface of the image-capturing element is divided based upon the image signal; an average ratio calculating unit that calculates a ratio of a pixel output average corresponding to another color to a pixel output average of a reference color for each partition area; an area extracting unit that extracts an area having a ratio of pixel output averages within a predetermined range; and a white balance adjustment unit that performs a white balance adjustment based upon pixel outputs corresponding to the individual colors from the extracted area.