Although deep network based solutions have been proposed for zero-shot recognition [1]}, [2]}, [3]}, to the best of our knowledge, we propose the first end-to-end trainable network for the ZSD problem that concurrently relates visual image features with the semantic label information. This network considers semantic embedding vector of classes as a fixed embedding within the network to produce prediction scores for both seen and unseen classes. We propose a novel loss formulation that incorporates max-margin learning [4]} and a semantic clustering loss based on class-scores of different meta-classes. While the max-margin loss tries to separate individual classes, semantic clustering loss tries to reduce the noise in semantic vectors by positioning similar classes together and dissimilar classes far apart. Notably, our proposed formulation assumes predefined unseen classes to explore the semantic relationships during model learning phase. This assumption is consistent with recent efforts in the literature which consider class semantics to solve the domain shift problem in ZSL [5]}, [6]} and does not a constitute transductive setting [7]}, [8]}, [9]}. Based on the premise that unseen class semantics may be unknown during training in several practical zero-shot scenarios, we also propose a variant of our approach that can be trained without predefined unseen classes.
Finally, we propose a comparison method for ZSD by extending a popular zero-shot recognition framework named ConSE [10]} using Faster-RCNN [11]}.
