Many problems with large-scale labeled training data have been impressively solved by deep learning. However, Unseen Class Categorization (UCC) with minimal information provided about target classes is the most commonly encountered setting in industry, which remains a challenging research problem in machine learning. Previous approaches to UCC either fail to generate a powerful discriminative feature extractor or fail to learn a flexible classifier that can be easily adapted to unseen classes. In this paper, we propose to address these issues through network reparameterization, \textit{i.e.}, reparametrizing the learnable weights of a network as a function of other variables, by which we decouple the feature extraction part and the classification part of a deep classification model to suit the special setting of UCC, securing both strong discriminability and excellent adaptability. Extensive experiments for UCC on several widely-used benchmark datasets in the settings of zero-shot and few-shot learning demonstrate that, our method with network reparameterization achieves state-of-the-art performance. The rich and accessible labeled data has fueled the revolutionary successes of deep learning in various tasks, e.g., visual recognition BID7 ), object detection BID20 ), machine translation BID1 ), etc. However, requiring numerous annotated data severely limits the applicability of deep learning algorithms to Unseen Class Categorization (UCC) for which we only have access to a limited amount of information, which is frequently encountered in industrial applications. Recently, an increasing number of approaches have been proposed to solve UCC with the help of either attribute descriptions (zero-shot learning (ZSL)) BID9 ; BID30 ) or one/a few labeled samples for each class (few-shot learning (FSL)) BID22 ; BID29 ).Previous approaches to UCC mainly have the following characteristics and limitations: (i) To obtain powerful discriminative feature representation, they often train a deep classification model employing state-of-the-art multi-class classification techniques. However, such models are hard to be adapted to new classes with limited supervision information due to the high volume of model parameters and the gradual updating scheme. (ii) To ensure the consistency of training and test settings and adaptability to new classes, previous methods often train a deep model in an episode fashion BID26 ), sometimes along with some specially designed meta-learning updating rules BID4 ). With episode-based training, the model acquires adaptability to new tasks after many training episodes using the knowledge it grasps during the training. However, the episode-based training strategy severely limits the model's capability of extracting discriminative features, because it does not fully exploit the diversity and variance of all classes within the training dataset. The trained model treats the classes in each episode as new classes and attempts to separate them. Therefore, it does not have memory of the competing information of these classes against all the other ones in the whole dataset beyond the current episode. Due to the neglect of this global (dataset-wise rather than episode-wise) discriminative information, the feature extraction capability of the model is suppressed, thus limiting the UCC performance.To address these issues, we propose to secure both powerful discriminability of feature extraction and strong adaptability of model classification through network reparameterization, i.e., reparametrizing the learnable weights of a network as a function of other variables. We decouple the feature extraction module and the classification module of a deep classification model, learn the former as a standard multi-class classification task to obtain a discriminative feature extractor, and learn the latter employing a light deep neural network that generates generic classification weights for unseen classes given limited exemplar information. We train the classification weight generator by following the episode-based training scheme to secure the adaptability. Our method can be flexibly applied to both ZSL and FSL, where the exemplar information about unseen classes are provided in the form of either the semantic attributes or one/a few labeled samples. Extensive experiments show that our proposed method achieves state-of-the-art performance on widely-used benchmark datasets for both tasks. In this paper, we propose a flexible framework for unseen class categorization with limited information provided about these classes. We secure two key factors, a powerful feature extractor and a flexible classifier, through network reparameterization. We decouple the feature extraction module and the classification module of a deep model for UCC. The feature extraction module is learned in a standard multi-class classification framework and the classification weight vector is generated by a network from exemplar information of the unseen classes. We train the classification weight generator in an episode-by-episode fashion to enable it flexibility for new tasks. Applying our framework for zero-shot learning (ZSL), we achieve much better results especially for the generalized ZSL setting than the state-of-the-art owing to our incorporation of inter-class separation information for learning the mapping from semantic space to visual space. For few-shot learning (FSL), we also achieve remarkable performance gains relative to existing methods due to the flexible scheme that make it possible a powerful feature extraction model and a flexible weight generation model.
A unified frame for both few-shot learning and zero-shot learning based on network reparameterization
{"src": "scitldr-AIC_train", "paper_id": "rJeyV2AcKX"}