1. A method of classifying at least one object of interest in a video, the method comprising: accessing, using at least one processing device, a frame of the video, the frame including at least one object of interest to be classified; performing, using the at least one processing device, object detection on the frame to detect the object of interest; tracking, using the at least one processing device, the object of interest over a plurality of frames in the video over time using a persistent tracking capability; isolating, using the at least one processing device, a segment of the frame that includes the object of interest; classifying, using the at least one processing device, the object of interest tracked in the tracking step, by processing the segment using deep learning; generating an output that indicates the classification of the object of interest; and continuing to track, using the classification of the object of interest, the object of interest after the object of interest has left and returned to a field of view of the video.