1. A method of real time camera relocalization comprising: receiving a sequence of depth map frames from a moving mobile depth camera each depth map frame comprising a plurality of image elements each having a depth value being related to a distance from the mobile depth camera to a surface in the scene captured by the mobile depth camera; tracking the position and orientation of the mobile depth camera using the depth map frames and simultaneously forming a 3D model of the environment in which the mobile depth camera is moving using the depth map frames; detecting a failure in the tracking of the position and orientation of the mobile depth camera; and relocalizing the mobile depth camera by recomputing its position and orientation using a current depth map captured by the mobile depth camera.