1. A system, comprising: a projector, that projects an optical pattern toward a surface; a camera, including a first plate with at least two off-axis apertures thereon, arranged to obtain an image of the optical pattern that has been projected, said camera being movable between different positions to image the surface from said different positions, said projector being at a specified angle of at least 5° relative to said camera; and a processor, receiving information from said camera, said information indicative of imaged scenes at different times, and said processor carrying out a first operation using information received through said apertures to determine a pose of said camera, and to determine three dimensional information about said surface based on a degree of deformation of said optical pattern on said surface indicative of a three dimensional surface, said pose and said three dimensional information being obtained from multiple different poses of said camera, and combining together said three dimensional information from said multiple poses using each of a plurality of poses to combine 3D information obtained from multiple different poses of the camera.