Nowadays, some researchers try to introduce Transformer to image restoration tasks. For exsample, Chen et al. proposed the Image Processing Transformer (IPT [1]}) which was pre-trained on large-scale datasets. In addition, contrastive learning is introduced for different image processing tasks. Therefore, the pre-trained model can efficiently be employed on the desired task after finetuning. However, IPT [1]} relies on large-scale datasets and has a large number of parameters (over 115.5M parameters), which greatly limits its application scenarios. To solve this issue, Liang et al. proposed the SwinIR [3]} for image restoration based on the Swin Transformer [4]}. Specifically, the Swin Transformer blocks (RSTB) is proposed for feature extraction and DIV2K+Flickr2K are used for training. Moreover, Lu et al. [5]} proposed an Efficient Super-Resolution Transformer (ESRT) for fast and accurate SISR. It is worth noting that ESRT is a lightweight model, which achieves competitive results with fewer parameters and low computing costs. Transformer is a powerful technology, but how to use fewer parameters and datasets to effectively train the model is still worth exploring.
