1. An image processing apparatus comprising a specific subject detection unit that detects a specific subject having a specific color from an assigned color image; a specific chromaticity calculation unit that calculates a specific chromaticity from the specific color representing the specific subject; a first white balance coefficient calculation unit that calculates a first white balance coefficient based on the color image; a second white balance coefficient calculation unit that calculates a second white balance coefficient based on the specific subject; a chromaticity evaluation unit that calculates a first chromaticity point by multiplying the calculated specific chromaticity by the first white balance coefficient, calculates a second chromaticity point by multiplying the calculated specific chromaticity by the second white balance coefficient, and evaluates the positional relationship between the first chromaticity point and the second chromaticity point in a color space; and a third white balance coefficient calculation unit that calculates a third white balance coefficient by weighting the first white balance coefficient and the second white balance coefficient based on a result of the evaluation by the chromaticity evaluation unit.