1. A system for object tracking, the system comprising: one or more processors and a non-transitory memory having instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of: detecting a location of a moving object in an image frame of a sequence of image frames using a motion-based object detection MogS module, thereby generating an object detection; for each image frame in the sequence of image frames, predicting the location of the moving object in the next image frame through object tracking using a Kalman filter prediction module, thereby generating a predicted object location; monitoring object tracking quality using a controller module; refining the predicted object location using a Kalman filter updating module; adjusting, by the controller module, the Kalman filter updating module according to a dissimilarity measure based on a spatial distance constraint and an appearance-based matching score, thereby improving object tracking quality and ameliorating gradual drifting of object tracking; and outputting a set of detected moving object locations in the sequence of image frames.