1. A method of maintaining a plurality of blob trackers for one or more video frames, the method comprising: associating a blob tracker with a blob generated for a video frame, wherein the blob includes pixels of at least a portion of one or more foreground objects in the video frame; determining a number of irregular size changes of the blob tracker over a threshold duration, wherein an irregular size change includes a change in size of the blob tracker between a previous video frame and the video frame greater than a size change threshold; determining the blob tracker is a second type of tracker based on the irregular size change or an irregular motion change over the threshold duration; and removing the blob tracker from the plurality of blob trackers maintained for the one or more video frames when the blob tracker is the second type of tracker.