1. A method for generating a bi-linear model of faces, comprising: acquiring, for each of a plurality of faces, a plurality of 2D images, the plurality of images acquired with a varying poses of the face and varying illuminations on the face; acquiring, for each of the plurality of faces, a 3D shape model; obtaining, for each of the plurality of faces, point-to-point correspondences using the 2D images and the 3D shape model; determining, for each of the plurality of faces, a diffuse texture from the 2D images; aligning the point-to-point correspondences and the diffuse textures into vectors; applying a singular value decomposition to the vectors to determine a joint variation of both shape and illumination; determining shape and illumination basis vectors from the shape and illumination; and reconstructing corresponding shape and illumination bases from a linear combination of the shape and illumination vectors, and a shape parameter to generate a bi-linear face model.