@inproceedings{
zheng2024saliencyguided,
title={Saliency-Guided Fine-Grained Temporal Mask Learning for Few-Shot Action Recognition},
author={Shuo Zheng and Yuanjie Dang and Peng Chen and Ruohong Huan and Dongdong Zhao and Ronghua Liang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=u7E4OzqBAo}
}
Temporal relation modeling is one of the core aspects of few-shot action recognition. Most previous works mainly focus on temporal relation modeling based on coarse-level actions, without considering the atomic action details and fine-grained temporal information. This oversight represents a significant limitation in this task. Specifically, coarse-level temporal relation modeling can make the few-shot models overfit in high-discrepancy temporal context, and ignore the low-discrepancy but high-semantic relevance action details in the video. To address these issues, we propose a saliency-guided fine-grained temporal mask learning method that models the temporal atomic action relation for few-shot action recognition in a finer manner. First, to model the comprehensive temporal relations of video instances, we design a temporal mask learning architecture to automatically search for the best matching of each atomic action snippet. Next, to exploit the low-discrepancy atomic action features, we introduce a saliency-guided temporal mask module to adaptively locate and excavate the atomic action information. After that, the few-shot predictions can be obtained by feeding the embedded rich temporal-relation features to a common feature matcher. Extensive experimental results on standard datasets demonstrate our method’s superior performance compared to existing state-of-the-art methods.
Saliency-Guided Fine-Grained Temporal Mask Learning for Few-Shot Action Recognition
Shuo Zheng Yuanjie Dang Peng Chen Ruohong Huan Dongdong Zhao Ronghua Liang