Discovering Object Masks with Transformers for Unsupervised Semantic Segmentation
The task of unsupervised semantic segmentation aims to cluster pixels into
semantically meaningful groups. Specifically, pixels assigned to the same
cluster should share high-level semantic properties like their object or part
category. This paper presents MaskDistill: a novel framework for unsupervised
semantic segmentation based on three key ideas. First, we advocate a
data-driven strategy to generate object masks that serve as a pixel grouping
prior for semantic segmentation. This approach omits handcrafted priors, which
are often designed for specific scene compositions and limit the applicability
of competing frameworks. Second, MaskDistill clusters the object masks to
obtain pseudo-ground-truth for training an initial object segmentation model.
Third, we leverage this model to filter out low-quality object masks. This
strategy mitigates the noise in our pixel grouping prior and results in a clean
collection of masks which we use to train a final segmentation model. By
combining these components, we can considerably outperform previous works for
unsupervised semantic segmentation on PASCAL (+11% mIoU) and COCO (+4% mask
AP50). Interestingly, as opposed to existing approaches, our framework does not
latch onto low-level image cues and is not limited to object-centric datasets.
The code and models will be made available.