1. A face component extraction apparatus, comprising: an eye detection unit which detects a plurality of combinations of eye regions, each combination forming a pair, in a face region in an image; a first calculation unit which calculates a first evaluation value for each pair of eye regions detected by the eye detection unit, the first calculation unit being related to likeness to eyes; a fitting unit which fits a plurality of extraction models for extracting a plurality of face components in the image based on a number of pairs of eye regions whose first evaluation values calculated by the first calculation unit are equal to or greater than a predetermined value; a second calculation unit which calculates a second evaluation value for each of a number of pairs of eye regions, the second evaluation value being related to a fitting condition of the plurality of extraction models which are fitted by the fitting unit; and a deciding unit which decides a fitting mode of the plurality of extraction models to be fitted by the fitting unit based on calculation results of a number of second evaluation values by the second calculation unit.