1. A method of one-pass video segmentation for differentiating between portions of video, the method comprising: obtaining, by a processor, two or more frames from a real-time video stream; building, by the processor, a background model using the two or more frames; labeling, by the processor, at least a portion of pixels in the two or more frames; and performing spatial or temporal filtering on the two or more frames by the processor.