1. A method comprising: identifying a portal in an image, wherein the portal is associated with a point of interest identified by a location of a first map marker in map data; receiving surface depth data identifying one or more distances from the geographic location of the camera that captured the image to objects of the image; determining a distance from the camera that captured the image to the portal in the image based on the surface depth data; determining, by one or more computing devices, an approximate geographic location of the portal based on the determined distance and the geographic location information and direction information associated with the image; adjusting, by the one or more computing devices, the location of the first map marker in the map data based on the approximate geographic location; and adjusting, by the one or more computing devices, the location of a second map marker in the map data based on the adjusted location of the first map marker.