6225988
A global optimal algorithm for class-dependent discretization of continuous data
This paper presents a new method to convert continuous variables into discrete variables for inductive machine learning. The method can be applied to pattern classification problems in machine learning and data mining. The discretization process is formulated as an optimization problem. We first use the normalized mutual information that measures the interdependence between the class labels and the variable to be discretized as the objective function, and then use fractional programming (iterative dynamic programming) to find its optimum. Unlike the majority of class-dependent discretization methods in the literature which only find the local optimum of the objective functions, the proposed method, OCDD, or Optimal Class-Dependent Discretization, finds the global optimum. The experimental results demonstrate that this algorithm is very effective in classification when coupled with popular learning systems such as C4.5 decision trees and Naive-Bayes classifier. It can be used to discretize continuous variables for many existing inductive learning systems.
16798311
Parameter Inference of Cost-Sensitive Boosting Algorithms
Several cost-sensitive boosting algorithms have been reported as effective methods in dealing with class imbalance problem. Misclassification costs, which reflect the different level of class identification importance, are integrated into the weight update formula of AdaBoost algorithm. Yet, it has been shown that the weight update parameter of AdaBoost is induced so as the training error can be reduced most rapidly. This is the most crucial step of AdaBoost in converting a weak learning algorithm into a strong one. However, most reported cost-sensitive boosting algorithms ignore such a property. In this paper, we come up with three versions of cost-sensitive AdaBoost algorithms where the parameters for sample weight updating are induced. Then, their identification abilities on the small classes are tested on four “real world” medical data sets taken from UCI Machine Learning Database based on F-measure. Our experimental results show that one of our proposed cost-sensitive AdaBoost algorithms is superior in achieving the best identification ability on the small class among all reported cost-sensitive boosting algorithms.
2919977
High-Order Pattern Discovery from Discrete-Valued Data
To uncover qualitative and quantitative patterns in a data set is a challenging task for research in the area of machine learning and data analysis. Due to the complexity of real-world data, high-order (polythetic) patterns or event associations, in addition to first-order class-dependent relationships, have to be acquired. Once the patterns of different orders are found, they should be represented in a form appropriate for further analysis and interpretation. The authors propose a novel method to discover qualitative and quantitative patterns (or event associations) inherent in a data set. It uses the adjusted residual analysis in statistics to test the significance of the occurrence of a pattern candidate against its expectation. To avoid exhaustive search of all possible combinations of primary events, techniques of eliminating the impossible pattern candidates are developed. The detected patterns of different orders are then represented in an attributed hypergraph which is lucid for pattern interpretation and analysis. Test results on artificial and real-world data are discussed toward the end of the paper.
12250963
Boosting for Learning Multiple Classes with Imbalanced Class Distribution
Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. This learning difficulty attracts a lot of research interests. Most efforts concentrate on bi-class problems. However, bi-class is not the only scenario where the class imbalance problem prevails. Reported solutions for bi-class applications are not applicable to multi-class problems. In this paper, we develop a cost-sensitive boosting algorithm to improve the classification performance of imbalanced data involving multiple classes. One barrier of applying the cost-sensitive boosting algorithm to the imbalanced data is that the cost matrix is often unavailable for a problem domain. To solve this problem, we apply Genetic Algorithm to search the optimum cost setup of each class. Empirical tests show that the proposed cost-sensitive boosting algorithm improves the classification performances of imbalanced data sets significantly.
23887431
Generative and Discriminative Learning by CL-Net
This correspondence presents a two-stage classification learning algorithm. The first stage approximates the class-conditional distribution of a discrete space using a separate mixture model, and the second stage investigates the class posterior probabilities by training a network. The first stage explores the generative information that is inherent in each class by using the Chow-Liu (CL) method, which approximates high-dimensional probability with a tree structure, namely, a dependence tree, whereas the second stage concentrates on discriminative learning to distinguish between classes. The resulting learning algorithm integrates the advantages of both generative learning and discriminative learning. Because it uses CL dependence-tree estimation, we call our algorithm CL-Net. Empirical tests indicate that the proposed learning algorithm makes significant improvements when compared with the related classifiers that are constructed by either generative learning or discriminative learning.
344886
From Association to Classification: Inference Using Weight of Evidence
Association and classification are two important tasks in data mining and knowledge discovery. Intensive studies have been carried out in both areas. But, how to apply discovered event associations to classification is still seldom found in current publications. Trying to bridge this gap, this paper extends our previous paper on significant event association discovery to classification. We propose to use weight of evidence to evaluate the evidence of a significant event association in support of, or against, a certain class membership. Traditional weight of evidence in information theory is extended here to measure the event associations of different orders with respect to a certain class. After the discovery of significant event associations inherent in a data set, it is easy and efficient to apply the weight of evidence measure for classifying an observation according to any attribute. With this approach, we achieve flexible prediction.
7385162
On the use of high-order ambiguity function for multi-component polynomial phase signals
Abstract Nonstationary signals appear often in real-life applications and many of them can be modeled as polynomial-phase signals (PPS). High-order ambiguity function (HAF) was first introduced to estimate the parameters of a single-component PPS, but has not been widely used for multi-component PPS because of its nonlinearity. Multi-component PPS arise for example, in Doppler radar applications when multiple objects are tracked simultaneously. In this paper, we show that HAF is virtually additive for multi-component PPS and suggest an algorithm to estimate their parameters. Numerical examples are presented to illustrate the theories.
6926567
Semi-Latent Dirichlet Allocation: A Hierarchical Model for Human Action Recognition
We propose a new method for human action recognition from video sequences using latent topic models. Video sequences are represented by a novel "bag-of-words" representation, where each frame corresponds to a "word". The major difference between our model and previous latent topic models for recognition problems in computer vision is that, our model is trained in a "semi-supervised" way. Our model has several advantages over other similar models. First of all, the training is much easier due to the decoupling of the model parameters. Secondly, it naturally solves the problem of how to choose the appropriate number of latent topics. Thirdly, it achieves much better performance by utilizing the information provided by the class labels in the training set. We present action classification and irregularity detection results, and show improvement over previous methods.
8649500
3D Non-rigid Surface Matching and Registration Based on Holomorphic Differentials
3D surface matching is fundamental for shape registration, deformable 3D non-rigid tracking, recognition and classification. In this paper we describe a novel approach for generating an efficient and optimal combined matching from multiple boundary-constrained conformal parameterizations for multiply connected domains (i.e., genus zero open surface with multiple boundaries), which always come from imperfect 3D data acquisition (holes, partial occlusions, change of pose and non-rigid deformation between scans). This optimality criterion is also used to assess how consistent each boundary is, and thus decide to enforce or relax boundary constraints across the two surfaces to be matched. The linear boundary-constrained conformal parameterization is based on the holomorphic differential forms, which map a surface with n boundaries conformally to a planar rectangle with (n - 2) horizontal slits, other two boundaries as constraints. The mapping is a diffeomorphism and intrinsic to the geometry, handles an open surface with arbitrary number of boundaries, and can be implemented as a linear system. Experimental results are given for real facial surface matching, deformable cloth non-rigid tracking, which demonstrate the efficiency of our method, especially for 3D non-rigid surfaces with significantly inconsistent boundaries.
40027798
Multiple Ranker Method in Document Retrieval
In this paper, we propose a multiple-ranker approach to make learning to rank methods more effective for document retrieval application. In traditional learning to rank methods, a ranker is learned from a set of queries together with their corresponding document rankings labeled by experts, and it is then used to predict the document rankings for new queries. But in practice, user queries vary in large diversity, which makes the single ranker learned from a close set of data not representative. The single ranker cannot be guaranteed with the best ranking result for every single query, and this becomes the bottleneck of traditional learning to rank approaches. To address this problem, we propose a multi-ranker approach. We train multiple diverse rankers which can cover diverse categories of queries, instead of an isolated one, and take an ensemble of these rankers for final prediction. We verify the proposed multipleranker approach over real-world datasets. The experimental results indicate that the proposed approach can outperform existing ‘learning to rank’ methods significantly.
31735857
Research on Vehicle Image Classifier Based on Concentration Regulating of Immune Clonal Selection
For time-variation and nonlinearity of vehicle image classifier design in complicated environment, a novel classifier based on Immune Clone Concentration Clustering Algorithm (ICCCA) was proposed, inspired by the clone selection theory in the natural immune system. It overcame the defects of traditional classification algorithms as multi-restraint and trap of local optimum. It also avoided the high complexity and regulating dissimilation caused by the cross and variation rules of exiting immune clone clustering algorithms. Directing towards two-class image pattern recognition problem, we defined the antibody and antigen of vehicle image, established class divisibility standard, and designed the affine mathematical model based on the antibody and antigen's Euclidean distance and their concentration. We proposed a method to create immune clone memory cell by the antibodies concentration as long as concentration regulating. The algorithm can guarantee the antibody diversity, and obtain the global optimal solution quickly and precisely. The experimental results verified that the classification accuracy of the algorithm is superior to other classification algorithm.