In this paper, a layer-wise mixture model (LMM) is developed to support hierarchical visual recognition, where a Bayesian approach is used to automatically adapt the visual hierarchy to the progressive improvements of the deep network along the time. Our LMM algorithm can provide an end-to-end approach for jointly learning: 1) the deep network for achieving more discriminative deep representations for object classes and their inter-class visual similarities; 2) the tree classifier for recognizing large numbers of object classes hierarchically; and 3) the visual hierarchy adaptation for achieving more accurate assignment and organization of large numbers of object classes. By learning the tree classifier, the deep network and the visual hierarchy adaptation jointly in an end-to-end manner, our LMM algorithm can achieve higher accuracy rates on hierarchical visual recognition. Our experiments are carried on ImageNet1K and ImageNet10K image sets, which have demonstrated that our LMM algorithm can achieve very competitive results on the accuracy rates as compared with the baseline methods.
One intuitive way for exploiting the inter-task relationships (inter-class visual similarities) is to integrate a tree structure to organize large numbers of object classes hierarchically, e.g., the tasks for training the classifiers for the fine-grained (visually-similar) object classes under the same parent node (in the same group) may have stronger inter-task relationships and share similar learning complexities. Such tree structures can be categorized into two types: (a) concept ontology @cite_6 @cite_21 @cite_11 @cite_2 @cite_5 ; and (b) label tree or visual hierarchy @cite_17 @cite_48 @cite_43 @cite_7 @cite_4 @cite_35 @cite_1 @cite_46 @cite_38 @cite_60 . It is worth noting that the feature space is the common space for classifier training and visual recognition @cite_24 , e.g., both classifier training and visual recognition are performed in the feature space rather than in the semantic label space. Thus it could be more attractive to organize large numbers of object classes hierarchically in the feature space according to their inter-class visual correlations.
@cite_35 @cite_38 @cite_4 @cite_7 @cite_60 @cite_48 @cite_21 @cite_1 @cite_17 @cite_6 @cite_24 @cite_43 @cite_2 @cite_5 @cite_46 @cite_11
2098728436 2153419563 2586505867 1987083125
When learning features for complex visual recognition problems, labeled image exemplars alone can be insufficient. While an object taxonomy specifying the categories' semantic relationships could bolster the learning process, not all relationships are relevant to a given visual classification task, nor does a single taxonomy capture all ties that are relevant. In light of these issues, we propose a discriminative feature learning approach that leverages multiple hierarchical taxonomies representing different semantic views of the object categories (e.g., for animal classes, one taxonomy could reflect their phylogenicties, while another could reflect their habitats). For each taxonomy, we first learn a tree of semantic kernels, where each node has a Mahalanobis kernel optimized to distinguish between the classes in its children nodes. Then, using the resulting semantic kernel forest, we learn class-specific kernel combinations to select only those relationships relevant to recognize each object class. To learn the weights, we introduce a novel hierarchical regularization term that further exploits the taxonomies' structure. We demonstrate our method on challenging object recognition datasets, and show that interleaving multiple taxonomic views yields significant accuracy improvements. We introduce an approach to learn discriminative visual representations while exploiting external semantic knowledge about object category relationships. Given a hierarchical taxonomy that captures semantic similarity between the objects, we learn a corresponding tree of metrics (ToM). In this tree, we have one metric for each non-leaf node of the object hierarchy, and each metric is responsible for discriminating among its immediate subcategory children. Specifically, a Mahalanobis metric learned for a given node must satisfy the appropriate (dis)similarity constraints generated only among its subtree members' training instances. To further exploit the semantics, we introduce a novel regularizer coupling the metrics that prefers a sparse disjoint set of features to be selected for each metric relative to its ancestor (supercategory) nodes' metrics. Intuitively, this reflects that visual cues most useful to distinguish the generic classes (e.g., feline vs. canine) should be different than those cues most useful to distinguish their component fine-grained classes (e.g., Persian cat vs. Siamese cat). We validate our approach with multiple image datasets using the WordNet taxonomy, show its advantages over alternative metric learning approaches, and analyze the meaning of attribute features selected by our algorithm. In this paper, a hierarchical deep multi-task learning (HD-MTL) algorithm is developed to support large-scale visual recognition (e.g., recognizing thousands or even tens of thousands of atomic object classes automatically). To achieve more effective accomplishment of the coarse-to-fine tasks for hierarchical visual recognition, multiple sets of deep features are first extracted from the different layers of deep convolutional neural networks (deep CNNs). A visual tree is then learned by assigning the visually-similar atomic object classes with similar learning complexities into the same group, and it can provide a good environment for identifying the inter-related learning tasks automatically. By leveraging the inter-task relatedness (inter-class similarities) to learn more discriminative group-specific deep representations, our deep multi-task learning algorithm can achieve the global optimum easily and obtain more discriminative node classifiers for distinguishing the visually-similar atomic object classes (in the same group) effectively. Our HD-MTL algorithm can control the inter-level error propagation effectively by using an end-to-end approach for jointly learning more representative deep CNNs (for image representation) and more discriminative tree classifier (for large-scale visual recognition) and updating them simultaneously. Our incremental deep learning algorithms can effectively adapt both the deep CNNs and the tree classifier to the new training images and the new object classes. Our experimental results have demonstrated that our HD-MTL algorithm can achieve very competitive results on both the accuracy rates and the computational efficiency for large-scale visual recognition. In this paper, a hierarchical multi-task structural learning algorithm is developed to support large-scale plant species identification, where a visual tree is constructed for organizing large numbers of plant species in a coarse-to-fine fashion and determining the inter-related learning tasks automatically. For a given parent node on the visual tree, it contains a set of sibling coarse-grained categories of plant species or sibling fine-grained plant species, and a multi-task structural learning algorithm is developed to train their inter-related classifiers jointly for enhancing their discrimination power. The inter-level relationship constraint, e.g., a plant image must first be assigned to a parent node (high-level non-leaf node) correctly if it can further be assigned to the most relevant child node (low-level non-leaf node or leaf node) on the visual tree, is formally defined and leveraged to learn more discriminative tree classifiers over the visual tree. Our experimental results have demonstrated the effectiveness of our hierarchical multi-task structural learning algorithm on training more discriminative tree classifiers for large-scale plant species identification.