BACKGROUND

Technical Field

The present invention relates to video recognition, and more particularly to an action recognition system for action recognition in unlabeled videos with domain adversarial learning and knowledge distillation.

Description of the Related Art

In machine learning, there exists the fundamental problem of domain adaptation when the source domain has abundant labeled training data and the target domain has no or little labeled training data but a massive amount of unlabeled data.

Accordingly, there is a need for a solution to the aforementioned problem relating to unlabeled data.

SUMMARY

According to an aspect of the present invention, an action recognition system is provided. The action recognition system includes a video capture device configured to capture a video sequence formed from a set of unlabeled testing video frames. The video sequence including one or more objects. The action recognition system further includes a processor. The processor is configured to pre-train a recognition engine formed from a reference set of convolutional neural networks (CNNs) on a still image domain that includes a set of labeled training still image frames. The processor is further configured to adapt the recognition engine to a video domain to form an adapted recognition engine, by applying a non-reference set of CNNs to a set of domains that include the still image domain, a synthetically degraded image domain, and the video domain. The synthetically degraded image domain includes labeled synthetically degraded versions of the labeled training still image frames included in the still image domain. The video domain includes random unlabeled training video frames. The processor is also configured to recognize, using the adapted recognition engine, an action performed by at least one of the objects in the video sequence to obtain a set of recognized actions. The processor is further configured to control a hardware device to perform a response action in response to an identification of an action type of an action performed by the at least one object. The identification of the action type is based on the set of recognized actions.

According to another aspect of the present invention, a computer-implemented method is provided for action recognition. The method includes capturing, by a video capture device, a video sequence formed from a set of unlabeled testing video frames. The video sequence includes one or more objects. The method further includes pre-training, by a processor, a recognition engine formed from a reference set of convolutional neural networks (CNNs) on a still image domain that includes a set of labeled training still image frames. The method also includes adapting, by the processor, the recognition engine to a video domain to form an adapted recognition engine, by applying a non-reference set of CNNs to a set of domains that include the still image domain, a synthetically degraded image domain, and the video domain. The synthetically degraded image domain includes labeled synthetically degraded versions of the labeled training still image frames included in the still image domain. The video domain includes random unlabeled training video frames. The method additionally includes recognizing, by the processor using the adapted recognition engine, an action performed by at least one of the objects in the video sequence to obtain a set of recognized actions. The processor further includes controlling, by the processor, a hardware device to perform a response action in response to an identification of an action type of an action performed by the at least one object. The identification of the action type is based on the set of recognized actions.

According to yet another aspect of the present invention, a computer program product is provided for action recognition. The computer program product includes a non-transitory computer readable storage medium having program instructions embodied therewith. The program instructions are executable by a computer to cause the computer to perform a method. The method includes capturing, by a video capture device, a video sequence formed from a set of unlabeled testing video frames. The video sequence includes one or more objects. The method further includes pre-training, by a processor, a recognition engine formed from a reference set of convolutional neural networks (CNNs) on a still image domain that includes a set of labeled training still image frames. The method also includes adapting, by the processor, the recognition engine to a video domain to form an adapted recognition engine, by applying a non-reference set of CNNs to a set of domains that include the still image domain, a synthetically degraded image domain, and the video domain. The synthetically degraded image domain includes labeled synthetically degraded versions of the labeled training still image frames included in the still image domain. The video domain includes random unlabeled training video frames. The method additionally includes recognizing, by the processor using the adapted recognition engine, an action performed by at least one of the objects in the video sequence to obtain a set of recognized actions. The method further includes controlling, by the processor, a hardware device to perform a response action in response to an identification of an action type of an action performed by the at least one object. The identification of the action type is based on the set of recognized actions.

These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.

