1. A white balance adjustment device for adjusting white balance of an input image, the white balance adjustment device comprising: dividing means for dividing an input image into a plurality of blocks; determining means for determining, from a color difference of each pixel included in each of the blocks, whether or not each pixel is of an object color; representative value calculating means for calculating a representative value for each of the blocks, from remaining pixels that are obtained by excluding, from pixels included in each of the blocks, pixels determined to be a pixel of an object color; and gain calculating means for calculating white balance gains from the representative values calculated for all the blocks.