1. A non-transitory computer accessible memory medium that stores program instructions for video holography, wherein the program instructions are executable by at least one processor to implement: receiving information regarding a two dimensional (2D) hogel array, comprising a plurality of hogel apertures, wherein the information regarding the 2D hogel array specifies number, size, and spacing of the hogel apertures; receiving information regarding a 3D scene to be rendered, including a scaling factor that maps the 3D scene to a 3D display volume; for each hogel of a sparse subset of the hogels in the 2D hogel array: computing a 2D perspective rendering of the 3D scene from the point of view (POV) of the hogel of the sparse subset; computing a color radiation intensity pattern based on the 2D perspective rendering; for each hogel of a complementary subset of the hogels with respect to the sparse subset: interpolating the color radiation intensity patterns of three or more hogels of the sparse subset in a neighborhood of the hogel, thereby generating an interpolated color radiation intensity pattern for the hogel of the complementary subset; computing a full set of color radiation intensity patterns for the 2D hogel array based on the color radiation intensity patterns of the sparse subset of hogels and the interpolated color radiation intensity patterns for the complementary subset of the hogels; and storing the full set of color radiation intensity patterns for the 2D hogel array, wherein the full set of color radiation intensity patterns for the 2D hogel array is useable to render a video holographic view of the 3D scene on a video display device.