1. A method for efficiently using memory in an autonomous localization and mapping system for a mobile device, the method comprising: observing features in the environment with a visual sensor coupled to the mobile device; receiving dead reckoning data from a sensor coupled to the mobile device; maintaining a plurality of landmarks for one or more maps in a first memory, where a landmark is referenced in a map in a global reference frame; copying data corresponding to the plurality of landmarks for the one or more maps from the first memory to a second memory, where the second memory operates faster than the first memory; using the data stored in the second memory when searching for matching landmarks; estimating a position for the mobile device within the global reference frame of the one or more maps at least partly in response to the observed features and the dead reckoning data; and selectively maintaining only a first portion of the plurality of landmarks for the one or more maps in the second memory and releasing a second portion such that memory resources previously allocated to the second portion can be returned to the system, where the first portion corresponds to landmarks that are likely to be in the same vicinity of the global reference frame as the position estimate for the mobile device.