1. A method comprising: receiving, by a computing system, a first image of a scene and a second image of the scene; determining a disparity loss function for each of a plurality of pixel locations in the first image, wherein the disparity loss function for a given pixel location in the first image indicates variation in color-space similarity between: (a) the pixel location in the first image and (b) a corresponding range of pixel locations in the second image; associating each of the plurality of pixel locations with one or more of a plurality of vertices in a vertex space, wherein the vertex space comprises two spatial dimensions and at least one color-space dimension; for each vertex in the vertex space, determining a vertex-disparity loss function based on the disparity loss functions corresponding to the associated pixel locations from the first image; applying the determined vertex-disparity loss functions to determine a respective disparity value for each of the plurality of vertices; and for each of the pixel locations in the first image, using the determined disparity value of each of one or more vertices that are associated with the pixel location as a basis to determine a disparity value for the pixel location in the first image.