1. A computer implemented method for tracking a moving object, comprising the steps of: registering a current image in a sequence of images with a previous image, in which the sequence of images is acquired of a scene by a moving camera, and in which the registering produces a registration result; tracking a moving object in the registered image to produces a tracking result, in which the registering and the tracking use a factorial hidden Markov model, in which the factorial hidden Markov model has a state variable for the current image, and further comprising: factoring the state variable into registration parameters and motion parameters; reregistering the registered current image with the previous image using the tracking result; and repeating the registering, tracking and registering steps by making the current image the previous image and a next image in the sequence of images the current image.