A review of heterogeneous data mining for brain disorder identification <s> Modeling feature correlations <s> DNA micro-arrays now permit scientists to screen thousands of genes simultaneously and determine whether those genes are active, hyperactive or silent in normal or cancerous tissue. Because these new micro-array devices generate bewildering amounts of raw data, new analytical methods must be developed to sort out whether cancer tissues have distinctive signatures of gene expression over normal tissues or other types of cancer tissues. ::: ::: In this paper, we address the problem of selection of a small subset of genes from broad patterns of gene expression data, recorded on DNA micro-arrays. Using available training examples from cancer and normal patients, we build a classifier suitable for genetic diagnosis, as well as drug discovery. Previous attempts to address this problem select genes with correlation techniques. We propose a new method of gene selection utilizing Support Vector Machine methods based on Recursive Feature Elimination (RFE). We demonstrate experimentally that the genes selected by our techniques yield better classification performance and are biologically relevant to cancer. ::: ::: In contrast with the baseline method, our method eliminates gene redundancy automatically and yields better and more compact gene subsets. In patients with leukemia our method discovered 2 genes that yield zero leave-one-out error, while 64 genes are necessary for the baseline method to get the best result (one leave-one-out error). In the colon cancer database, using only 4 genes our method is 98% accurate, while the baseline method is only 86% accurate. <s> BIB001 </s> A review of heterogeneous data mining for brain disorder identification <s> Modeling feature correlations <s> This paper aims to take general tensors as inputs for supervised learning. A supervised tensor learning (STL) framework is established for convex optimization based learning techniques such as support vector machines (SVM) and minimax probability machines (MPM). Within the STL framework, many conventional learning machines can be generalized to take n/sup th/-order tensors as inputs. We also study the applications of tensors to learning machine design and feature extraction by linear discriminant analysis (LDA). Our method for tensor based feature extraction is named the tenor rank-one discriminant analysis (TR1DA). These generalized algorithms have several advantages: 1) reduce the curse of dimension problem in machine learning and data mining; 2) avoid the failure to converge; and 3) achieve better separation between the different categories of samples. As an example, we generalize MPM to its STL version, which is named the tensor MPM (TMPM). TMPM learns a series of tensor projections iteratively. It is then evaluated against the original MPM. Our experiments on a binary classification problem show that TMPM significantly outperforms the original MPM. <s> BIB002 </s> A review of heterogeneous data mining for brain disorder identification <s> Modeling feature correlations <s> Classifying objects that are sampled jointly from two or more domains has many applications. The tensor product feature space is useful for modeling interactions between feature sets in different domains but feature selection in the tensor product feature space is challenging. Conventional feature selection methods ignore the structure of the feature space and may not provide the optimal results. In this paper we propose methods for selecting features in the original feature spaces of different domains. We obtained sparsity through two approaches, one using integer quadratic programming and another using L1-norm regularization. Experimental studies on biological data sets validate our approach. <s> BIB003 </s> A review of heterogeneous data mining for brain disorder identification <s> Modeling feature correlations <s> In the era of big data, we can easily access information from multiple views which may be obtained from different sources or feature subsets. Generally, different views provide complementary information for learning tasks. Thus, multi-view learning can facilitate the learning process and is prevalent in a wide range of application domains. For example, in medical science, measurements from a series of medical examinations are documented for each subject, including clinical, imaging, immunologic, serologic and cognitive measures which are obtained from multiple sources. Specifically, for brain diagnosis, we can have different quantitative analysis which can be seen as different feature subsets of a subject. It is desirable to combine all these features in an effective way for disease diagnosis. However, some measurements from less relevant medical examinations can introduce irrelevant information which can even be exaggerated after view combinations. Feature selection should therefore be incorporated in the process of multi-view learning. In this paper, we explore tensor product to bring different views together in a joint space, and present a dual method of tensor-based multi-view feature selection DUAL-TMFS based on the idea of support vector machine recursive feature elimination. Experiments conducted on datasets derived from neurological disorder demonstrate the features selected by our proposed method yield better classification performance and are relevant to disease diagnosis. <s> BIB004 </s> A review of heterogeneous data mining for brain disorder identification <s> Modeling feature correlations <s> With rapidly growing amount of data available on the web, it becomes increasingly likely to obtain data from different perspectives for multi-view learning. Some successive examples of web applications include recommendation and target advertising. Specifically, to predict whether a user will click an ad in a query context, there are available features extracted from user profile, ad information and query description, and each of them can only capture part of the task signals from a particular aspect/view. Different views provide complementary information to learn a practical model for these applications. Therefore, an effective integration of the multi-view information is critical to facilitate the learning performance. In this paper, we propose a general predictor, named multi-view machines (MVMs), that can effectively explore the full-order interactions between features from multiple views. A joint factorization is applied for the interaction parameters which makes parameter estimation more accurate under sparsity and renders the model with the capacity to avoid overfitting. Moreover, MVMs can work in conjunction with different loss functions for a variety of machine learning tasks. The advantages of MVMs are illustrated through comparison with other methods for multi-view prediction, including support vector machines (SVMs), support tensor machines (STMs) and factorization machines (FMs). A stochastic gradient descent method and a distributed implementation on Spark are presented to learn the MVM model. Through empirical studies on two real-world web application datasets, we demonstrate the effectiveness of MVMs on modeling feature interactions in multi-view data. A 3.51\% accuracy improvement is shown on MVMs over FMs for the problem of movie rating prediction, and 0.57\% for ad click prediction. <s> BIB005
One of the key issues for multi-view classification is to choose an appropriate tool to model features and their correlations hidden in multiple views, since this directly determines how information will be used. In contrast to modeling on views, another direction for modeling multiview data is to directly consider the correlations between features from multiple views. Since taking the tensor product of their respective feature spaces corresponds to the interaction of features from multiple views, the concept of tensor serves as a backbone for incorporating multi-view features into a consensus representation by means of tensor product, where the complex multiple relationships among views are embedded within the tensor structures. By mining structural information contained in the tensor, knowledge of multi-view features can be extracted and used to establish a predictive model. Smalter et al. formulate the problem of feature selection in the tensor product space as an integer quadratic programming problem BIB003 . However, this method is computationally intractable on many views, since it directly selects features in the tensor product space resulting in the curse of dimensionality, as the method (b) shown in Fig. 6 . Cao et al. propose to use a tensor-based approach to model features and their correlations hidden in the original multi-view data BIB004 . The operation of tensor product can be used to bring m-view feature vectors of each instance together, leading to a tensorial representation for common structure across multiple views, and allowing us to adequately diffuse relationships and encode information among multi-view features. In this manner, the multi-view classification task is essentially transformed from an independent domain of each view to a consensus domain as a tensor classification problem. By using X i to denote Q m v¼1 x ðvÞ i , the dataset of labeled multi-view instances can be represented as D ¼ fðX i ; y i Þg n i¼1 . Note that each multi-view instance X i is an mth-order tensor that lies in the tensor product space R I 1 ÂÁÁÁÂI m . Based on the definitions of inner product and tensor norm, multi-view classification can be formulated as a global convex optimization problem in the framework of supervised tensor learning BIB002 . This model is named as multi-view SVM BIB004 , and it can be solved with the use of optimization techniques developed for SVM. Furthermore, a dual method for multi-view feature selection is proposed in BIB004 that leverages the relationship between original multi-view features and reconstructed tensor product features to facilitate the implementation of feature selection, as the method (c) in Fig. 6 . It is a wrapper model which selects useful features in conjunction with the classifier and simultaneously exploits the correlations among multiple views. Following the idea of SVM-based recursive feature elimination BIB001 , multi-view feature selection is consistently formulated and implemented in the framework of multi-view SVM. This idea can extend to include lower order feature interactions and to employ a variety of loss functions for classification or regression BIB005 .