A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Variable and feature selection have become the focus of much research in areas of application for which datasets with tens or hundreds of thousands of variables are available. These areas include text processing of internet documents, gene expression array analysis, and combinatorial chemistry. The objective of variable selection is three-fold: improving the prediction performance of the predictors, providing faster and more cost-effective predictors, and providing a better understanding of the underlying process that generated the data. The contributions of this special issue cover a wide range of aspects of such problems: providing a better definition of the objective function, feature construction, feature ranking, multivariate feature selection, efficient search methods, and feature validity assessment methods. <s> BIB001 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> We propose new methods to evaluate variable subset relevance with a view to variable selection. Relevance criteria are derived from Support Vector Machines and are based on weight vector ||w||2 or generalization error bounds sensitivity with respect to a variable. Experiments on linear and non-linear toy problems and real-world datasets have been carried out to assess the effectiveness of these criteria. Results show that the criterion based on weight vector derivative achieves good results and performs consistently well over the datasets we used. <s> BIB002 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Relief algorithms are general and successful attribute estimators. They are able to detect conditional dependencies between attributes and provide a unified view on the attribute estimation in regression and classification. In addition, their quality estimates have a natural interpretation. While they have commonly been viewed as feature subset selection methods that are applied in prepossessing step before a model is learned, they have actually been used successfully in a variety of settings, e.g., to select splits or to guide constructive induction in the building phase of decision or regression tree learning, as the attribute weighting method and also in the inductive logic programming. ::: ::: A broad spectrum of successful uses calls for especially careful investigation of various features Relief algorithms have. In this paper we theoretically and empirically investigate and discuss how and why they work, their theoretical and practical properties, their parameters, what kind of dependencies they detect, how do they scale up to large number of examples and features, how to sample data for them, how robust are they regarding the noise, how irrelevant and redundant attributes influence their output and how different metrics influences them. <s> BIB003 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> DNA micro-arrays now permit scientists to screen thousands of genes simultaneously and determine whether those genes are active, hyperactive or silent in normal or cancerous tissue. Because these new micro-array devices generate bewildering amounts of raw data, new analytical methods must be developed to sort out whether cancer tissues have distinctive signatures of gene expression over normal tissues or other types of cancer tissues. ::: ::: In this paper, we address the problem of selection of a small subset of genes from broad patterns of gene expression data, recorded on DNA micro-arrays. Using available training examples from cancer and normal patients, we build a classifier suitable for genetic diagnosis, as well as drug discovery. Previous attempts to address this problem select genes with correlation techniques. We propose a new method of gene selection utilizing Support Vector Machine methods based on Recursive Feature Elimination (RFE). We demonstrate experimentally that the genes selected by our techniques yield better classification performance and are biologically relevant to cancer. ::: ::: In contrast with the baseline method, our method eliminates gene redundancy automatically and yields better and more compact gene subsets. In patients with leukemia our method discovered 2 genes that yield zero leave-one-out error, while 64 genes are necessary for the baseline method to get the best result (one leave-one-out error). In the colon cancer database, using only 4 genes our method is 98% accurate, while the baseline method is only 86% accurate. <s> BIB004 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Feature selection is an important problem for pattern classification systems. We study how to select good features according to the maximal statistical dependency criterion based on mutual information. Because of the difficulty in directly implementing the maximal dependency condition, we first derive an equivalent form, called minimal-redundancy-maximal-relevance criterion (mRMR), for first-order incremental feature selection. Then, we present a two-stage feature selection algorithm by combining mRMR and other more sophisticated feature selectors (e.g., wrappers). This allows us to select a compact set of superior features at very low cost. We perform extensive experimental comparison of our algorithm and other methods using three different classifiers (naive Bayes, support vector machine, and linear discriminate analysis) and four different data sets (handwritten digits, arrhythmia, NCI cancer cell lines, and lymphoma tissues). The results confirm that mRMR leads to promising improvement on feature selection and classification accuracy. <s> BIB005 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Support Vector Machines have proved to be powerful tools for classification tasks combining the minimization of classification errors and maximizing their generalization capabilities. Feature selection, however, is not considered explicitly in the basic model formulation. We propose a linearly penalized Support Vector Machines (LP-SVM) model where feature selection is performed simultaneously with model construction. Its application to a problem of customer retention and a comparison with other feature selection techniques demonstrates its effectiveness. <s> BIB006 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Various form features affect consumer preference regarding product design. It is, therefore, important that designers identify these critical form features to aid them in developing appealing products. However, the problems inherent in choosing product form features have not yet been intensively investigated. In this paper, an approach based on multiclass support vector machine recursive feature elimination (SVM-RFE) is proposed to streamline the selection of optimum product form features. First, a one-versus-one (OVO) multiclass fuzzy support vector machines (multiclass fuzzy SVM) model using a Gaussian kernel was constructed based on product samples from mobile phones. Second, an optimal training model parameter set was determined using two-step cross-validation. Finally, a multiclass SVM-RFE process was applied to select critical form features by either using overall ranking or class-specific ranking. The weight distribution of each iterative step can be used to analyze the relative importance of each of the form features. The results of our experiment show that the multiclass SVM-RFE process is not only very useful for identifying critical form features with minimum generalization errors but also can be used to select the smallest feature subset for building a prediction model with a given discrimination capability. <s> BIB007 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Recent advances in Multiple Kernel Learning (MKL) have positioned it as an attractive tool for tackling many supervised learning tasks. The development of efficient gradient descent based optimization schemes has made it possible to tackle large scale problems. Simultaneously, MKL based algorithms have achieved very good results on challenging real world applications. Yet, despite their successes, MKL approaches are limited in that they focus on learning a linear combination of given base kernels. In this paper, we observe that existing MKL formulations can be extended to learn general kernel combinations subject to general regularization. This can be achieved while retaining all the efficiency of existing large scale optimization algorithms. To highlight the advantages of generalized kernel learning, we tackle feature selection problems on benchmark vision and UCI databases. It is demonstrated that the proposed formulation can lead to better results not only as compared to traditional MKL but also as compared to state-of-the-art wrapper and filter methods for feature selection. <s> BIB008 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> This paper studies the general problem of learning kernels based on a polynomial combination of base kernels. We analyze this problem in the case of regression and the kernel ridge regression algorithm. We examine the corresponding learning kernel optimization problem, show how that minimax problem can be reduced to a simpler minimization problem, and prove that the global solution of this problem always lies on the boundary. We give a projection-based gradient descent algorithm for solving the optimization problem, shown empirically to converge in few iterations. Finally, we report the results of extensive experiments with this algorithm using several publicly available datasets demonstrating the effectiveness of our technique. <s> BIB009 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> We introduce a novel wrapper Algorithm for Feature Selection, using Support Vector Machines with kernel functions. Our method is based on a sequential backward selection, using the number of errors in a validation subset as the measure to decide which feature to remove in each iteration. We compare our approach with other algorithms like a filter method or Recursive Feature Elimination SVM to demonstrate its effectiveness and efficiency. <s> BIB010 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> To reveal and leverage the correlated and complemental information between different views, a great amount of multi-view learning algorithms have been proposed in recent years. However, unsupervised feature selection in multi-view learning is still a challenge due to lack of data labels that could be utilized to select the discriminative features. Moreover, most of the traditional feature selection methods are developed for the single-view data, and are not directly applicable to the multi-view data. Therefore, we propose an unsupervised learning method called Adaptive Unsupervised Multi-view Feature Selection (AUMFS) in this paper. AUMFS attempts to jointly utilize three kinds of vital information, i.e., data cluster structure, data similarity and the correlations between different views, contained in the original data together for feature selection. To achieve this goal, a robust sparse regression model with the l2,1-norm penalty is introduced to predict data cluster labels, and at the same time, multiple view-dependent visual similar graphs are constructed to flexibly model the visual similarity in each view. Then, AUMFS integrates data cluster labels prediction and adaptive multi-view visual similar graph learning into a unified framework. To solve the objective function of AUMFS, a simple yet efficient iterative method is proposed. We apply AUMFS to three visual concept recognition applications (i.e., social image concept recognition, object recognition and video-based human action recognition) on four benchmark datasets. Experimental results show the proposed method significantly outperforms several state-of-the-art feature selection methods. More importantly, our method is not very sensitive to the parameters and the optimization method converges very fast. <s> BIB011 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Mining discriminative features for graph data has attracted much attention in recent years due to its important role in constructing graph classifiers, generating graph indices, etc. Most measurement of interestingness of discriminative subgraph features are defined on certain graphs, where the structure of graph objects are certain, and the binary edges within each graph represent the "presence" of linkages among the nodes. In many real-world applications, however, the linkage structure of the graphs is inherently uncertain. Therefore, existing measurements of interestingness based upon certain graphs are unable to capture the structural uncertainty in these applications effectively. In this paper, we study the problem of discriminative subgraph feature selection from uncertain graphs. This problem is challenging and different from conventional subgraph mining problems because both the structure of the graph objects and the discrimination score of each subgraph feature are uncertain. To address these challenges, we propose a novel discriminative subgraph feature selection method, DUG, which can find discriminative subgraph features in uncertain graphs based upon different statistical measures including expectation, median, mode and phi-probability. We first compute the probability distribution of the discrimination scores for each subgraph feature based on dynamic programming. Then a branch-and-bound algorithm is proposed to search for discriminative subgraphs efficiently. Extensive experiments on various neuroimaging applications (i.e., Alzheimer's Disease, ADHD and HIV) have been performed to analyze the gain in performance by taking into account structural uncertainties in identifying discriminative subgraph features for graph classification. <s> BIB012 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> In many data mining applications, we often face the problem of cross-domain learning, i.e., to transfer the already learned knowledge from a source domain to a target domain. In particular, this problem becomes very challenging when there is no or little labeled training data available in the target domain, which is not an uncommon scenario as it is expensive and in certain cases even impossible to obtain any labeled training data in the target domain in many real world applications. In the literature, though few efforts are reported to attempt to solve this challenging problem, the solutions are all rather limited making this problem still open and challenging. On the other hand, as it is not uncommon to face this problem in many applications, an effective solution to this problem shall generate substantial societal impacts. In this paper, we address this problem and propose a new framework, called DISMUTE, taking advantage of the typically available multiple views of the data in domains. Consequently, DISMUTE is based on discriminative feature selection for multi-view cross-domain learning. Theoretic analysis and extensive evaluations in the specific application of object identification and image classification against several state-of-the-art methods demonstrate the outstanding superiority of DISMUTE. <s> BIB013 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> Combining information from various data sources has become an important research topic in machine learning with many scientific applications. Most previous studies employ kernels or graphs to integrate different types of features, which routinely assume one weight for one type of features. However, for many problems, the importance of features in one source to an individual cluster of data can be varied, which makes the previous approaches ineffective. In this paper, we propose a novel multi-view learning model to integrate all features and learn the weight for every feature with respect to each cluster individually via new joint structured sparsity-inducing norms. The proposed multi-view learning framework allows us not only to perform clustering tasks, but also to deal with classification tasks by an extension when the labeling knowledge is available. A new efficient algorithm is derived to solve the formulated objective with rigorous theoretical proof on its convergence. We applied our new data fusion method to five broadly used multi-view data sets for both clustering and classification. In all experimental results, our method clearly outperforms other related state-of-the-art methods. <s> BIB014 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> To better understand, search, and classify image and video information, many visual feature descriptors have been proposed to describe elementary visual characteristics, such as the shape, the color, the texture, etc. How to integrate these heterogeneous visual features and identify the important ones from them for specific vision tasks has become an increasingly critical problem. In this paper, We propose a novel Sparse Multimodal Learning (SMML) approach to integrate such heterogeneous features by using the joint structured sparsity regularizations to learn the feature importance of for the vision tasks from both group-wise and individual point of views. A new optimization algorithm is also introduced to solve the non-smooth objective with rigorously proved global convergence. We applied our SMML method to five broadly used object categorization and scene understanding image data sets for both single-label and multi-label image classification tasks. For each data set we integrate six different types of popularly used image features. Compared to existing scene and object categorization methods using either single modality or multi-modalities of features, our approach always achieves better performances measured. <s> BIB015 </s> A review of heterogeneous data mining for brain disorder identification <s> Multi-view feature analysis <s> In the era of big data, we can easily access information from multiple views which may be obtained from different sources or feature subsets. Generally, different views provide complementary information for learning tasks. Thus, multi-view learning can facilitate the learning process and is prevalent in a wide range of application domains. For example, in medical science, measurements from a series of medical examinations are documented for each subject, including clinical, imaging, immunologic, serologic and cognitive measures which are obtained from multiple sources. Specifically, for brain diagnosis, we can have different quantitative analysis which can be seen as different feature subsets of a subject. It is desirable to combine all these features in an effective way for disease diagnosis. However, some measurements from less relevant medical examinations can introduce irrelevant information which can even be exaggerated after view combinations. Feature selection should therefore be incorporated in the process of multi-view learning. In this paper, we explore tensor product to bring different views together in a joint space, and present a dual method of tensor-based multi-view feature selection DUAL-TMFS based on the idea of support vector machine recursive feature elimination. Experiments conducted on datasets derived from neurological disorder demonstrate the features selected by our proposed method yield better classification performance and are relevant to disease diagnosis. <s> BIB016
Medical science witnesses everyday measurements from a series of medical examinations documented for each subject, including clinical, imaging, immunologic, serologic, and cognitive measures  , as shown in Fig. 5 . Each group of measures characterizes the health state of a subject from different aspects. This type of data is named as multi-view data, and each group of measures form a distinct view quantifying subjects in one specific feature space. Therefore, it is critical to combine them to improve the learning performance, while simply concatenating features from all views and transforming a multi-view data into a single-view data, as the method (a) shown in Fig. 6 , would fail to leverage the underlying correlations between different views. Representative methods for multi-view learning can be categorized into three groups: co-training, multiple kernel learning, and subspace learning  . Generally, the cotraining style algorithm is a classic approach for semi-supervised learning, which trains in alternation to maximize the mutual agreement on different views. Multiple kernel learning algorithms combine kernels that naturally correspond to different views, either linearly  or nonlinearly BIB008 BIB009 to improve learning performance. Subspace learning algorithms learn a latent subspace, from which multiple views are generated. Multiple kernel learning and subspace learning are generalized as co-regularization style algorithms  , where the disagreement between the functions of different views is taken as a part of the objective function to be minimized. Overall, by exploring the consistency and complementary properties of different views, multi-view learning is more effective than singleview learning. In the multi-view setting for brain disorders, or for medical studies in general, a critical problem is that there may be limited subjects available (i.e., a small n) yet introducing a large number of measurements (i.e., a large P m i¼1 I i ). Within the multi-view data, not all features in different views are relevant to the learning task, and some irrelevant features may introduce unexpected noise. The irrelevant information can even be exaggerated after view combinations thereby degrading performance. Therefore, it is necessary to take care of feature selection in the learning process. Feature selection results can also be used by researchers to find biomarkers for brain diseases. Such biomarkers are clinically imperative for detecting injury to the brain in the earliest stage before it is irreversible. Valid biomarkers can be used to aid diagnosis, monitor disease progression, and evaluate effects of intervention BIB012 . Conventional feature selection approaches can be divided into three main directions: filter, wrapper, and embedded methods BIB001 . Filter methods compute a discrimination score of each feature independently of the other features based on the correlation between the feature and the label, e.g., information gain, Gini index, Relief BIB005 BIB003 . Wrapper methods measure the usefulness of feature subsets according to their predictive power, optimizing the subsequent induction procedure that uses the respective subset for classification BIB016 BIB004 BIB002 BIB007 BIB010 . Embedded methods perform feature selection in the process of model training based on sparsity regularization BIB011 BIB013 BIB014 BIB015 . For example, Miranda et al. add a regularization term that penalizes the size of the selected feature subset to the standard cost function of SVM, thereby optimizing the new objective function to conduct feature selection BIB006 . Essentially, the process of feature selection and learning algorithm interact in embedded methods which means the learning part and the feature selection part cannot be separated, while wrapper methods utilize the learning algorithm as a black box. However, directly applying these feature selection approaches to each separate view would fail to leverage multi-view correlations. By taking into account the latent interactions among views and the redundancy triggered by multiple views, it is desirable to combine multi-view data in a principled manner and perform feature selection to obtain consensus and discriminative low-dimensional feature representations.