1. A method of tracking a pose of a moving camera, comprising the steps of: receiving a first received digitized image from a camera, said image comprising a plurality of intensities corresponding to a domain of points on a 2-dimensional grid, and including one or more landmarks for tracking; receiving a sequence of digitized images from said camera, each image comprising a plurality of intensities corresponding to a domain of points on a 2-dimensional grid; recording, for each of said sequence of digitized images, the pose and 2D correspondences of each of said landmarks; reconstructing a location and appearance of a 2-dimensional texture patch from the 2D correspondences of the landmarks of said sequence of digitized images by triangulation and optimization; computing back-projection errors by comparing said reconstructed texture patch with said first received image; and reconstructing said location and appearance of said 2-dimensional texture patch from the 2D correspondences of the landmarks of said sequence of digitized images by triangulation and optimization after eliminating those landmarks with large back-projection errors.