1. A method for determining three dimensional (3D) shape, comprising: capturing with a camera a plurality of images of an object in motion; derive a general relation that relates spatial and temporal image derivatives to bidirectional reflectance distribution function (BRDF) derivatives; exploiting rank deficiency to eliminate BRDF terms and recover depth or normal for directional lighting; and using depth-normal-BRDF relation to recover depth or normal for unknown arbitrary lightings; and determining the 3D shape from the motion.