CIGMO: Categorical invariant representations in a deep generative framework
Data of general object images have two most common structures: (1) each
object of a given shape can be rendered in multiple different views, and (2)
shapes of objects can be categorized in such a way that the diversity of shapes
is much larger across categories than within a category. Existing deep
generative models can typically capture either structure, but not both. In this
work, we introduce a novel deep generative model, called CIGMO, that can learn
to represent category, shape, and view factors from image data. The model is
comprised of multiple modules of shape representations that are each
specialized to a particular category and disentangled from view representation,
and can be learned using a group-based weakly supervised learning method. By
empirical investigation, we show that our model can effectively discover
categories of object shapes despite large view variation and quantitatively
supersede various previous methods including the state-of-the-art invariant
clustering algorithm. Further, we show that our approach using
category-specialization can enhance the learned shape representation to better
perform down-stream tasks such as one-shot object identification as well as
shape-view disentanglement.