1. A computer-implemented method to select planes that are used to render a navigable three-dimensional scene from a collection of images stored in an electronic database, the computer-implemented method comprising: receiving, by a computer processor, the collection of images from the electronic database; extracting, by the computer processor, scene information from the collection of images; detecting, by the computer processor, vanishing directions within each image in the collection of images using edge detection and two-dimensional line segment extraction to form orthogonal triplets having at least two orthogonal vanishings directions; identifying, by the computer processor, each plane in the navigable three-dimensional scene based on the vanishing directions and the scene information; and generating, by the computer processor, a Markov Random Field distribution to assign pixels to each plane based on minimization functions applied to the Markov Random Field distribution to optimize a global cost of assigning the pixel to a plane in the navigable three-dimensional scene, wherein a portion of the global cost of plane-pixel assignments is represented by smoothness terms included in the minimization functions applied to a Markov Random Field distribution graph, the smoothness terms are calculated to prefer transitions between planes at pixels corresponding to discontinuities at two-dimensional lines that are consistent with the vanishing directions orthogonal to the normal vector of an occluding plane.