@inproceedings{
xiao2024asymmetric,
title={Asymmetric Event-Guided Video Super-Resolution},
author={Zeyu Xiao and Dachun Kai and Yueyi Zhang and Xiaoyan Sun and Zhiwei Xiong},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=zVgZfHRM3g}
}
Event cameras are novel bio-inspired cameras that record asynchronous events with high temporal resolution and dynamic range. Leveraging the auxiliary temporal information recorded by event cameras holds great promise for the task of video super-resolution (VSR). However, existing event-guided VSR methods assume that the event and RGB cameras are strictly calibrated (e.g., pixel-level sensor designs in DAVIS 240/346). This assumption proves limiting in emerging high-resolution devices, such as dual-lens smartphones and unmanned aerial vehicles, where such precise calibration is typically unavailable. To unlock more event-guided application scenarios, we propose to perform the task of asymmetric event-guided VSR for the first time, and we propose an Asymmetric Event-guided VSR Network (AsEVSRN) for this new task. AsEVSRN incorporates two specialized designs for leveraging the asymmetric event stream in VSR. Firstly, the content hallucination module dynamically enhances event and RGB information by exploiting their complementary nature, thereby adaptively boosting representational capacity. Secondly, the event-enhanced bidirectional recurrent cells align and propagate temporal features fused with features from content-hallucinated frames. Within the bidirectional recurrent cells, event-enhanced flow is employed for simultaneous utilization and fusion of temporal information at both the feature and pixel levels. Comprehensive experimental results affirm that our method consistently produces superior results both quantitatively and qualitatively. Code will be released.
Asymmetric Event-Guided Video Super-Resolution
Zeyu Xiao Dachun Kai Yueyi Zhang Xiaoyan Sun Zhiwei Xiong