GSMFlow: Generation Shifts Mitigating Flow for Generalized Zero-Shot Learning
Generalized Zero-Shot Learning (GZSL) aims to recognize images from both the
seen and unseen classes by transferring semantic knowledge from seen to unseen
classes. It is a promising solution to take the advantage of generative models
to hallucinate realistic unseen samples based on the knowledge learned from the
seen classes. However, due to the generation shifts, the synthesized samples by
most existing methods may drift from the real distribution of the unseen data.
To address this issue, we propose a novel flow-based generative framework that
consists of multiple conditional affine coupling layers for learning unseen
data generation. Specifically, we discover and address three potential problems
that trigger the generation shifts, i.e., semantic inconsistency, variance
collapse, and structure disorder. First, to enhance the reflection of the
semantic information in the generated samples, we explicitly embed the semantic
information into the transformation in each conditional affine coupling layer.
Second, to recover the intrinsic variance of the real unseen features, we
introduce a boundary sample mining strategy with entropy maximization to
discover more difficult visual variants of semantic prototypes and hereby
adjust the decision boundary of the classifiers. Third, a relative positioning
strategy is proposed to revise the attribute embeddings, guiding them to fully
preserve the inter-class geometric structure and further avoid structure
disorder in the semantic space. Extensive experimental results on four GZSL
benchmark datasets demonstrate that GSMFlow achieves the state-of-the-art
performance on GZSL.