1. A system comprising: one or more processors; an image sensor; a projector; one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising: capturing one or more images of an environment using the image sensor; determining a location of a display medium within the environment based at least in part on the one or more images, wherein the display medium has a known parametric description; projecting content onto the display medium using the projector; determining an observed depth map of at least a portion of the environment that contains the display medium, wherein the observed depth map includes first depth information associated with the display medium and an object within the environment that at least partially occludes the display medium; computing a display depth map of the display medium based at least in part on the location and the known parametric description of the display medium, wherein the display depth map includes second depth information associated with the display medium, the second depth information including a first depth for a first point of the display medium that is unobscured in the one or more images and a second depth for a second point of the display medium that is obscured by the object and that is not depicted in the one or more images; and producing a foreground depth map based at least in part on differences between the second depth information of the display depth map and the first depth information of the observed depth map, the foreground depth map indicating a difference in depth between a third point associated with the observed depth map and a corresponding third point associated with the display depth map.