1. A method, comprising: receiving a speech input; generating at least two speech recognition candidates from the speech input; observing a scene related to the speech input using one or more non-acoustic sensors; segmenting the observed scene into a plurality of regions, wherein each of the regions corresponds to an object or a surface in the observed scene; computing properties for at least a given region of the plurality of regions, wherein computing the properties for the given region comprises computing one or more characteristics for the given region and computing one or more relationships between the given region and remaining ones of the plurality of regions, and wherein the one or more characteristics of the given region comprise a color, a shape and a textual label; and selecting one of the speech recognition candidates based at least in part on the computed properties of the given region.