@inproceedings{
wu2024compacter,
title={Compacter: A Lightweight Transformer for Image Restoration},
author={Zhijian Wu and Jun Li and Yang Hu and Dingjiang Huang},
booktitle={ACM Multimedia 2024},
year={2024},
url={https://openreview.net/forum?id=gvrDYlQXxw}
}
Although deep learning-based methods have made significant advances in the field of image restoration (IR), they often suffer from excessive model parameters. To tackle this problem, this work proposes a compact Transformer (Compacter) for lightweight image restoration by making several key designs. We employ the concepts of projection sharing, adaptive interaction, and heterogeneous aggregation to develop a novel Compact Adaptive Self-Attention (CASA). Specifically, CASA utilizes shared projection to generate Query, Key, and Value to simultaneously model spatial and channel-wise self-attention. The adaptive interaction process is then used to propagate and integrate global information from two different dimensions, thus enabling omnidirectional relational interaction. Finally, a depth-wise convolution is incorporated on Value to complement heterogeneous local information, enabling global-local coupling. Moreover, we propose a Dual Selective Gated Module (DSGM) to dynamically encapsulate the globality into each pixel for context-adaptive aggregation. Extensive experiments demonstrate that our Compacter achieves state-of-the-art performance for a variety of lightweight IR tasks with approximately 400K parameters.
Compacter: A Lightweight Transformer for Image Restoration
Zhijian Wu Jun Li Yang Hu Dingjiang Huang