1. A non-transitory computer readable memory medium storing programming instructions executable by a processor to: receive, from at least one camera, one or more images of a physical environment of a user of a three-dimensional (3D) display system; determine a data imagery map of the physical environment based at least in part on the one or more images, wherein the one or more images of the physical environment of the user includes the user; receive instructions to render a virtual 3D scene on a display of the 3D display system; and incorporate the data imagery map of the physical environment, including a rendering of a reflection of the user on a 3D object within the 3D scene, into the virtual 3D scene, thereby incorporating physical world conditions into virtual world imagery.