1. A computer-implemented method for generating a sparse representation of a light field, comprising: receiving the light field captured as a plurality of images depicting a scene; receiving depth estimates of points in the scene; determining an error between a reconstruction from the depth estimates and the received images; and storing the depth estimates and the error.