1. An apparatus for providing an augmented view of a physical environment comprising: a panoramic camera; a processor; and a memory coupled to the processor and configured to store instructions for an augmented reality application, the augmented reality application including: a capture process that when executed is operable to capture a panorama of the physical environment using the panoramic camera, the panorama having a panoramic field of view and being embodied as a sequence of video frames, an initialization process that when executed is operable to determine an initial pose of the panoramic camera, a tracking process that when executed is operable to track the panoramic camera to update the initial pose to subsequent poses to account for movement of the panoramic camera about the physical environment, the tracking process to utilize features dispersed across the panoramic field of view of the video frames, and an augmentation process that when executed is operable to augment the panorama by merging computer-generated elements with the panorama based on the updated pose of the panoramic camera, and to display a sub-portion of the panorama, along with the computer-generated elements, as an augmented view, the displayed augmented view having a field of view that is less than the panoramic field of view.