1. A method of generating a lookup table for use in estimating spatial depth in a visual scene, comprising: at a server having one or more processors, and memory storing one or more programs configured for execution by the one or more processors: identifying a plurality of distinct subsets of IR illuminators of a camera system that has a 2-dimensional array of image sensors and a plurality of IR illuminators in fixed locations relative to the array of image sensors; partitioning the image sensors into a plurality of pixels; for each pixel, and for each of m distinct depths from the respective pixel: simulating a virtual surface at the respective depth; for each of the plurality of distinct subsets of IR illuminators, determining an expected IR light intensity at the respective pixel based on the respective depth and based on only the respective subset of IR illuminators emitting IR light; forming an intensity vector using the expected IR light intensities for each of the distinct subsets; and normalizing the intensity vector; and for each pixel, constructing a lookup table comprising the normalized vectors corresponding to the pixel, wherein the lookup table associates each respective normalized vector with the respective depth of the respective simulated surface.