1. A recognition apparatus for recognizing a facial portion of a person containing a face by taking a sequence of color image data, the recognition apparatus comprising: an image conversion unit that converts single color image data including the facial portion into single grayscale image data; a facial portion-identifying unit that performs a first process that identifies a facial image region of the single grayscale image data, and performs a second process that identifies a facial image region of the single color image data when the first process fails to identify the facial image region; an acquisition unit that obtains skin color information about skin color of the person from pixel values within a region of the single color image data corresponding to the facial image region identified by the first process; and a storage unit that stores the obtained skin color information as stored skin color information; wherein the facial portion-identifying unit performs the second process that identifies the facial image region based on a center of gravity of color, the center of gravity of color being calculated based on the stored skin color information, the stored skin color information being obtained from another single color image data that was converted into another single color grayscale image data and processed by the first process before the single image data.