1. A computer-implemented method for tracking objects determined to be in one or more video frames, the method comprising: generating, by a video analytics system, a plurality of blobs for a video frame from the one or more video frames, the plurality of blobs including pixels representing a plurality of objects in the video frame, wherein the plurality of objects include a foreground object and a background object, wherein the video analytics system generates a first blob for the background object due to detecting a change in pixels associated with the background object; determining a number of the plurality of blobs generated for the video frame; determining a size of a smallest blob generated for the video frame; determining a size threshold based on the number of the plurality of blobs and the size of the smallest blob; determining that the first blob is less than the size threshold; filtering, based on the first blob being less than the size threshold, the first blob from the plurality of blobs, wherein filtering the first blob removes the first blob from the plurality of blobs; and tracking the plurality of blobs that remain after filtering.