@inproceedings{
ma2024learning,
title={Learning Cross-Spectral Prior for Image Super-Resolution},
author={Chenxi Ma and Weimin Tan and Shili Zhou and Bo Yan},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=pffPeV7GNR}
}
With the rising interest in multi-camera cross-spectral systems, cross-spectral images have been widely used in computer vision and image processing. Therefore, an effective super-resolution (SR) method is significant in providing high-resolution (HR) cross-spectral images for different research and applications. However, existing SR methods rarely consider utilizing cross-spectral information to assist the SR of visible images and cannot handle the complex degradation (noise, high brightness, low light) and misalignment problem in low-resolution (LR) cross-spectral images. Here, we first explore the potential of using near-infrared (NIR) image guidance for better SR, based on the observation that NIR images can preserve valuable information for recovering adequate image details. To take full advantage of the cross-spectral prior, we propose a novel $\textbf{C}$ross-$\textbf{S}$pectral $\textbf{P}$rior guided image $\textbf{SR}$ approach ($\textbf{CSPSR}$). Concretely, we design a cross-view matching (CVM) module and a dynamic multi-modal fusion (DMF) module to enhance the spatial correlation between cross-spectral images and to bridge the multi-modal feature gap, respectively. The DMF module facilitates adaptive feature adaptation and effective information transmission through a dynamic convolution and a cross-spectral feature transfer (CSFT) unit. Extensive experiments demonstrate the effectiveness of our CSPSR, which can exploit the prominent cross-spectral information to produce state-of-the-art results.
Learning Cross-Spectral Prior for Image Super-Resolution
Chenxi Ma Weimin Tan Shili Zhou Bo Yan