Abstract Noisy labeled data represent a rich source of information that often are easily accessible and cheap to obtain, but label noise might also have many negative consequences if not accounted for. How to fully utilize noisy labels has been studied extensively within the framework of standard supervised machine learning over a period of several decades. However, very little research has been conducted on solving the challenge posed by noisy labels in non-standard settings. This includes situations where only a fraction of the samples are labeled (semi-supervised) and each high-dimensional sample is associated with multiple labels. In this work, we present a novel semi-supervised and multi-label dimensionality reduction method that effectively utilizes information from both noisy multi-labels and unlabeled data. With the proposed Noisy multi-label semi-supervised dimensionality reduction (NMLSDR) method, the noisy multi-labels are denoised and unlabeled data are labeled simultaneously via a specially designed label propagation algorithm. NMLSDR then learns a projection matrix for reducing the dimensionality by maximizing the dependence between the enlarged and denoised multi-label space and the features in the projected space. Extensive experiments on synthetic data, benchmark datasets, as well as a real-world case study, demonstrate the effectiveness of the proposed algorithm and show that it outperforms state-of-the-art multi-label feature extraction algorithms.
@cite_77 , a supervised method for doing DR based on dependence maximization @cite_62 called Multi-label dimensionality reduction via dependence maximization (MDDM) was introduced. MDDM attempts to maximize the feature-label dependence using the Hilbert-Schmidt independence criterion and was originally formulated in two different ways. MDDMp is based on orthonormal projection directions, whereas MDDMf makes the projected features orthonormal. showed that MDDMp can be formulated using least squares and added a PCA term to the cost function in a new method called Multi-label feature extraction via maximizing feature variance and feature-label dependence simultaneously (MVMD) @cite_25 .
@cite_77 @cite_62 @cite_25
2253239179 2124982040 2762016725 2614937618
We derive a least-squares formulation for MDDMp technique.A novel multi-label feature extraction algorithm is proposed.Our algorithm maximizes both feature variance and feature-label dependence.Experiments show that our algorithm is a competitive candidate. Dimensionality reduction is an important pre-processing procedure for multi-label classification to mitigate the possible effect of dimensionality curse, which is divided into feature extraction and selection. Principal component analysis (PCA) and multi-label dimensionality reduction via dependence maximization (MDDM) represent two mainstream feature extraction techniques for unsupervised and supervised paradigms. They produce many small and a few large positive eigenvalues respectively, which could deteriorate the classification performance due to an improper number of projection directions. It has been proved that PCA proposed primarily via maximizing feature variance is associated with a least-squares formulation. In this paper, we prove that MDDM with orthonormal projection directions also falls into the least-squares framework, which originally maximizes Hilbert-Schmidt independence criterion (HSIC). Then we propose a novel multi-label feature extraction method to integrate two least-squares formulae through a linear combination, which maximizes both feature variance and feature-label dependence simultaneously and thus results in a proper number of positive eigenvalues. Experimental results on eight data sets show that our proposed method can achieve a better performance, compared with other seven state-of-the-art multi-label feature extraction algorithms. Multi-label learning deals with data associated with multiple labels simultaneously. Like other machine learning and data mining tasks, multi-label learning also suffers from the curse of dimensionality. Although dimensionality reduction has been studied for many years, multi-label dimensionality reduction remains almost untouched. In this paper, we propose a multi-label dimensionality reduction method, MDDM, which attempts to project the original data into a lower-dimensional feature space maximizing the dependence between the original feature description and the associated class labels. Based on the Hilbert-Schmidt Independence Criterion, we derive a closed-form solution which enables the dimensionality reduction process to be efficient. Experiments validate the performance of MDDM. Like other machine learning paradigms, multi-label learning also suffers from the curse of dimensionality problem. Multi-label dimensionality reduction can alleviate the problem but they generally ask for sufficient labeled samples. Nevertheless, we often may only have scarce labeled samples and abundant unlabeled samples. In this paper, we propose a @math emi-supervised @math ulti-label @math imensionality @math eduction based on @math ependence @math aximization approach (SMDRdm in short). SDMRdm assumes the semantic similarity and feature similarity of multi-label samples are inter-depended. SMDRdm first applies label propagation on a neighborhood graph composed with labeled and unlabeled samples to obtain the soft labels of unlabeled samples, and then measures the semantic similarity between all the training samples (including unlabeled ones) based on these soft labels and available labels of labeled samples. Next, it measures the feature similarity between samples in the subspace projected by the target projective matrix, instead of the original high-dimensional feature space. After that, it maximizes the dependence between these two types of similarities and incorporates the dependence into linear discriminant analysis to optimize the target projective matrix. Experiments on publicly accessible multi-label data sets demonstrate that SMDRdm achieves more prominent results than other related approaches across various evaluation metrics. In addition, the empirical study also shows the semantic similarity between samples derived from soft labels works better than that derived from scarce available labels. Abstract Linear discriminant analysis (LDA) is one of the most popular single-label (multi-class) feature extraction techniques. For multi-label case, two slightly different generalized versions have been introduced independently. We argue whether there exists a framework to unify such two multi-label LDA methods and to derive more well-performed multi-label LDA techniques further. In this paper, we build a weighted multi-label LDA framework (wMLDA) to consolidate two existing multi-label LDA-type methods with binary and correlation-based weight forms, and further collect two additional weight forms with entropy and fuzzy principles. To exploit both label and feature information more sufficiently, via maximizing dependence based on Hilbert–Schmidt independence criterion, a novel dependence-based weight form is proposed, which is formulated as a non-convex quadratic programing problem with l 1 -norm and non-negative constraints and then is solved by random block coordinate descent method with a linear convergence rate. Experiments on ten data sets illustrate that our dependence-based wMLDA works the best, and five wMLDA-type algorithms are superior to canonical correlation analysis and multi-label dimensionality reduction via dependency maximization, according to five multi-label classification performance measures and Wilcoxon statistical test.