1. A system, comprising: a projector that projects an optical pattern toward a surface at an angle of projection; a camera, including a first plate with at least two off-axis apertures thereon, arranged to obtain an image of the optical pattern that has been projected, said camera being movable between different positions to image the surface from said different positions, said projector and said angle of projection being at a specified angle of at least 30° relative to said camera; and a processor that receives information from said camera, said information indicative of imaged scenes at different times, wherein said processor uses information received through said apertures to determine a relationship, based on the angle of projection of the optical pattern, between different amounts of deformation of the projected pattern and a feature of the surface, to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface, said pose and said three dimensional information obtained from multiple different poses of said camera, and wherein said processor combines together said three dimensional information from said multiple poses using each of a plurality of poses.