1. A method for visually tracking a moving real-world object by estimating a three dimensional position and a two dimensional velocity of the moving object, the method comprising: capturing a first image at a camera at a first time; processing an image region of the first captured image to obtain first visual cues representing first input features of the moving object to be tracked, the image region in the first image being the region the moving object is expected to be located at the first time; initializing a 2D tracker template based on the first input features; capturing a second image at the camera at a second time subsequent to the first time; processing the second captured image to obtain second visual cues representing second input features; at a processor, determining a two dimensional position of the moving object to be tracked from the second captured image by estimating parameters of a two dimensional transformation on the determined object in the first captured image or the first input features that best match the determined object in the second captured image or the second input features; determining a two dimensional velocity of the object to be tracked based on the position of the moving object in the first and second captured images; estimating a depth of the moving object to be tracked at the first time based on a third cue obtained at the first time; estimating a change of the depth of the moving object to be tracked between the first time and the second time based on the third cue and a fourth cue obtained at the second time; determining a relative change of size of the moving object to be tracked from the first captured image and the determined object from the second captured image by performing a two dimensional transformation on the determined object in the first and second captured images or the first and second input features; updating the estimated depth and the change of the depth of the moving object to be tracked based on the relative change of the size of the moving object to be tracked; combining the updated depth of the moving object to be tracked, the change of the depth of the moving object to be tracked, two dimensional position of the moving object to be tracked, and the two dimensional velocity of the moving object to be tracked to obtain a three dimensional coordinate of the object to be tracked at the second time; and outputting the three dimensional coordinate of the object to be tracked and tracking the moving object.