1. A method for automatically performing age estimation based on the facial image of people using the notion of pairwise facial image and relative age, comprising the following steps of: a) generating first pairwise facial images along with reference faces from a database of facial images, and annotating the first pairwise facial images for their relative ages and the reference faces for their absolute ages, b) determining face-based class similarity for the first pairwise facial images by appearance-based face clusters or demographic categories, c) training learning machines using the first pairwise facial images so that the learning machines estimate the relative age and face-based class similarity score of any pairwise facial images, d) constructing second pairwise facial images from an input face and the reference faces, e) estimating the relative ages of the second pairwise facial images, using the trained learning machines, and f) estimating the age of the input face using the relative ages of the second pairwise facial images, wherein each of the learning machines represents a face-based class among pre-determined face-based classes, wherein first face of each pair in the pairwise facial images belongs to the face-based class, and wherein the face-based class similarity represents whether two faces in the pairwise facial images belong to one of predetermined face-based classes.