1. A method of producing a three-dimensional (3D) synthetic viewpoint of a scene, the method comprising: receiving an image of the scene from a single local camera, the image including at least one vertical object; analyzing the image to determine a context of the scene and to extract at least one surface feature of the scene; developing an initial set of planar world coordinates for the image received from the single local camera based on the at least one surface feature to estimate an initial local perspective projection matrix; mapping image pixels of the image to a world plane to produce a synthetic viewpoint of the scene based on a 3D construction of the scene by using the initial local perspective projection matrix to map image space coordinates to world coordinates, wherein the synthetic viewpoint is a selectable synthetic viewpoint that is coincident with a user viewpoint of the single local camera; removing vertical distortion associated with the at least one vertical object by: computing world coordinates of an intersection of the at least one vertical object with a surface corresponding to the surface feature using the initial local perspective projection matrix, and based on the world coordinates of the intersection and an estimated height of the at least one vertical object, projecting the vertical object onto a vertical plane, wherein computing the world coordinates of the intersection includes: estimating image space coordinates of first and second features of the at least one vertical object, the first feature corresponding to the intersection of the at least one vertical object with the surface, estimating a midpoint of the first feature in world coordinates, and mapping the estimated image space coordinates of the first feature into world coordinates of the first feature using the initial local perspective projection matrix; and in-painting the at least one vertical object on a horizontal plane with estimated background pixels.