758563
Cost-sensitive boosting for classification of imbalanced data
Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. The significant difficulty and frequent occurrence of the class imbalance problem indicate the need for extra research efforts. The objective of this paper is to investigate meta-techniques applicable to most classifier learning algorithms, with the aim to advance the classification of imbalanced data. The AdaBoost algorithm is reported as a successful meta-technique for improving classification accuracy. The insight gained from a comprehensive analysis of the AdaBoost algorithm in terms of its advantages and shortcomings in tacking the class imbalance problem leads to the exploration of three cost-sensitive boosting algorithms, which are developed by introducing cost items into the learning framework of AdaBoost. Further analysis shows that one of the proposed algorithms tallies with the stagewise additive modelling in statistics to minimize the cost exponential loss. These boosting algorithms are also studied with respect to their weighting strategies towards different types of samples, and their effectiveness in identifying rare cases through experiments on several real world medical data sets, where the class imbalance problem prevails.
12250963
Boosting for Learning Multiple Classes with Imbalanced Class Distribution
Classification of data with imbalanced class distribution has posed a significant drawback of the performance attainable by most standard classifier learning algorithms, which assume a relatively balanced class distribution and equal misclassification costs. This learning difficulty attracts a lot of research interests. Most efforts concentrate on bi-class problems. However, bi-class is not the only scenario where the class imbalance problem prevails. Reported solutions for bi-class applications are not applicable to multi-class problems. In this paper, we develop a cost-sensitive boosting algorithm to improve the classification performance of imbalanced data involving multiple classes. One barrier of applying the cost-sensitive boosting algorithm to the imbalanced data is that the cost matrix is often unavailable for a problem domain. To solve this problem, we apply Genetic Algorithm to search the optimum cost setup of each class. Empirical tests show that the proposed cost-sensitive boosting algorithm improves the classification performances of imbalanced data sets significantly.
10246554
An Overview of Associative Classifiers
Associative classification is a new classification approach integrating association mining and classification. It becomes a significant tool for knowledge discovery and data mining. This paper presents a critical overview of certain aspects on modelling associative classifiers including event association mining, computational complexity in mining associations, methods of constructing associative classifiers based on the association patterns discovered, some similarities and differences among the reported associative classification systems, and experimental comparisons of these associative classifiers with the conventional classification system such as C4.5.
344886
From Association to Classification: Inference Using Weight of Evidence
Association and classification are two important tasks in data mining and knowledge discovery. Intensive studies have been carried out in both areas. But, how to apply discovered event associations to classification is still seldom found in current publications. Trying to bridge this gap, this paper extends our previous paper on significant event association discovery to classification. We propose to use weight of evidence to evaluate the evidence of a significant event association in support of, or against, a certain class membership. Traditional weight of evidence in information theory is extended here to measure the event associations of different orders with respect to a certain class. After the discovery of significant event associations inherent in a data set, it is easy and efficient to apply the weight of evidence measure for classifying an observation according to any attribute. With this approach, we achieve flexible prediction.
15440093
Pattern discovery: a data driven approach to decision support
Decision support nowadays is more and more targeted to large scale complicated systems and domains. The success of a decision support system relies mainly on its capability of processing large amounts of data and efficiently extracting useful knowledge from the data, especially knowledge which is previously unknown to the decision makers. With a large scale system, traditional knowledge acquisition models become inefficient and/or more biased, due to the subjectivity of the experts or the pre-assumptions of certain ideas or algorithmic procedures. Today, with the rapid development of computer technologies, the capability of collecting data has been greatly advanced. Data becomes the most valuable resource for an organization. We present a fundamental framework toward intelligent decision support by analyzing a large amount of mixed-mode data (data with a mixture of continuous and categorical values) in order to bridge the subjectivity and objectivity of a decision support process. By considering significant associations of artifacts (events) inherent in the data as patterns, we define patterns as statistically significant associations among feature values represented by joint events or hypercells in the feature space. We then present an algorithm which automatically discovers statistically significant hypercells (patterns) based on: 1) a residual analysis, which tests the significance of the deviation when the occurrence of a hypercell differs from its expectation, and 2) an optimization formulation to enable recursive discovery. By discovering patterns from data sets based on such an objective measure, the nature of the problem domain will be revealed. The patterns can then be applied to solve specific problems as being interpreted or inferred with.
22289409
Attribute Clustering for Grouping, Selection, and Classification of Gene Expression Data
18710459
Conjugate and natural gradient rules for BYY harmony learning on Gaussian mixture with automated model selection
Under the Bayesian Ying–Yang (BYY) harmony learning theory, a harmony function has been developed on a BI-directional architecture of the BYY system for Gaussian mixture with an important feature that, via its maximization through a general gradient rule, a model selection can be made automatically during parameter learning on a set of sample data from a Gaussian mixture. This paper further proposes the conjugate and natural gradient rules to efficiently implement the maximization of the harmony function, i.e. the BYY harmony learning, on Gaussian mixture. It is demonstrated by simulation experiments that these two new gradient rules not only work well, but also converge more quickly than the general gradient ones.
9587091
Topology control for delay-constraint data collection in wireless sensor networks
Data collection is one of the most important operations in wireless sensor networks. Many practical applications require the real-time data transmission, such as monitoring, tracking, etc. In this paper, we import and define the topology control problem for delay-constraint data collection (TDDC), and then formalize this problem into an integer programming problem. As NP-Hardness of this problem, we present a load-aware power-increased topology control algorithm (namely LPTC) to heuristically solve the problem. The theoretical analysis shows that this algorithm can reach O(1)-approximation ratio for the linear networks. And we also analyze the impact of the delay-constraint on the worst-case for the planar networks. Moreover, this paper designs two localized algorithms, called as SDEL and DDEL, based on the area division for TDDC problem. The experimental results show that LPTC algorithm can save at least 17% power consumptions compared with HBH algorithm in many situations.
1208928
Enforcing convexity for improved alignment with constrained local models
Constrained local models (CLMs) have recently demonstrated good performance in non-rigid object alignment/ tracking in comparison to leading holistic approaches (e.g., AAMs). A major problem hindering the development of CLMs further, for non-rigid object alignment/tracking, is how to jointly optimize the global warp update across all local search responses. Previous methods have either used general purpose optimizers (e.g., simplex methods) or graph based optimization techniques. Unfortunately, problems exist with both these approaches when applied to CLMs. In this paper, we propose a new approach for optimizing the global warp update in an efficient manner by enforcing convexity at each local patch response surface. Furthermore, we show that the classic Lucas-Kanade approach to gradient descent image alignment can be viewed as a special case of our proposed framework. Finally, we demonstrate that our approach receives improved performance for the task of non-rigid face alignment/tracking on the MultiPIE database and the UNBC-McMaster archive.
3021341
Parallel multi-teraflops studies of the magnetic structure of FeMn alloys
Massively parallel computers have been used to perform first principles spin dynamics (SD) simulations of the magnetic structure of /spl gamma/-FeMn (iron-manganese). The code uses a novel parallel approach to solve the Kohn-Sham equations obtained from density functional theory (DFT). This approach uses limited local communications avoiding the large global communications found in other DFT based methods. This allows our code to scale efficiently to very large processor counts and large system sizes. Computationally our method can be implemented using mainly BLAS3 routines which typically run at a very high percentage of peak speed on most computers. Our large scale quantum mechanical simulations using this method reveal new details of the orientational configuration of the magnetic moments of /spl gamma/-FeMn that are unobtainable by any other means. The understanding of the magnetic structure of /spl gamma/-FeMn is important in exchange bias, which involves the use of an antiferromagnetic (AFM) layer such as FeMn to pin the orientation of the magnetic moment of a proximate ferromagnetic (FM). Exchange bias is of fundamental importance in magnetic multilayer storage and read head devices. Here the equation of motion of first principles SD is used to perform relaxations of model magnetic structures to the true ground (equilibrium) state. Our code has achieved a maximum execution rate of 2.46 teraflops on the IBM SP at the National Energy Research Scientific Computing Center (NERSC) and 4.65 teraflops on the HP-Compaq machine at the Pittsburgh Supercomputing Center (PSC).
1200220
Boosting over Groups and Its Application to Acronym-Expansion Extraction
In many real-world classification applications, instances are generated from different `groups'. Take webpage classification as an example, the webpages for training and testing can be naturally grouped by network domains, which often vary a lot from one to another in domain size or webpage template. The differences between `groups' would result that the distribution of instances from different `groups' also vary. Thus, it is not so reasonable to equally treat the instances as the independent elements during training and testing as in conventional classification algorithms. This paper addresses the classification problem where all the instances can be naturally grouped. Specifically, we give a formulation to this kind of problem and propose a simple but effective boosting approach, which is called AdaBoost.Group. The problem is demonstrated by the task of recognizing acronyms and their expansions from text, where all the instances are grouped by sentences. The experimental results show that our approach is more appropriate to this kind of problems than conventional classification approaches.