blackLow-Resolution RGB Pixels (LR-RGB).
RGB pixels are the most intuitive priors for inpainting. To address quadratically costly computation of transformer, ICT [1]} leverages LR-RGB priors \(\mathbf {\tilde{P}}_{rgb}\)  (\(32\times 32\)  in our implementation) to guide the inpainting. To further reduce the color space, all RGB combinations are clustered to 512 discrete K-Means centers on the whole ImageNet as [2]}. Moreover, masked pixels are replaced with trainable tokens [MASK]. Then generated \(\mathbf {\tilde{P}}_{rgb}\)  are bilinear upsampled to the stage-2 model. Thus stage-2 can be also seen as a super-resolution task, while stage-1 is working as LR inpainting. Since the color space is clustered to 512 discrete centers, the optimization can be seen as a classification. Therefore, the objective of \(\mathbf {\tilde{P}}_{rgb}\)  is to minimize the negative log-likelihood with cross-entropy (CE) for masked tokens.
