@inproceedings{
choi2024understanding,
title={Understanding Multi-Granularity for Open-Vocabulary Part Segmentation},
author={Jiho Choi and Seonho Lee and Seungho Lee and Minhyun Lee and Hyunjung Shim},
booktitle={The Thirty-eighth Annual Conference on Neural Information Processing Systems},
year={2024},
url={https://openreview.net/forum?id=hE6ZxU0N3c}
}
Open-vocabulary part segmentation (OVPS) is an emerging research area focused on segmenting fine-grained entities using diverse and previously unseen vocabularies.
Our study highlights the inherent complexities of part segmentation due to intricate boundaries and diverse granularity, reflecting the knowledge-based nature of part identification.
To address these challenges, we propose PartCLIPSeg, a novel framework utilizing generalized parts and object-level contexts to mitigate the lack of generalization in fine-grained parts.
PartCLIPSeg integrates competitive part relationships and attention control, alleviating ambiguous boundaries and underrepresented parts.
Experimental results demonstrate that PartCLIPSeg outperforms existing state-of-the-art OVPS methods, offering refined segmentation and an advanced understanding of part relationships within images.
Through extensive experiments, our model demonstrated a significant improvement over the state-of-the-art models on the Pascal-Part-116, ADE20K-Part-234, and PartImageNet datasets.
Our code is available at https://github.com/kaist-cvml/part-clipseg.
Understanding Multi-Granularity for Open-Vocabulary Part Segmentation
Jiho Choi Seonho Lee Seungho Lee Minhyun Lee Hyunjung Shim