A Survey on Soft Subspace Clustering <s> Scalable Learning for Large Dataset <s> Large-margin methods, such as support vector machines (SVMs), have been very successful in classification problems. Recently, maximum margin discriminant analysis (MMDA) was proposed that extends the large-margin idea to feature extraction. It often outperforms traditional methods such as kernel principal component analysis (KPCA) and kernel Fisher discriminant analysis (KFD). However, as in the SVM, its time complexity is cubic in the number of training points m, and is thus computationally inefficient on massive data sets. In this paper, we propose an (1 + isin)2-approximation algorithm for obtaining the MMDA features by extending the core vector machine. The resultant time complexity is only linear in m, while its space complexity is independent of m. Extensive comparisons with the original MMDA, KPCA, and KFD on a number of large data sets show that the proposed feature extractor can improve classification accuracy, and is also faster than these kernel-based methods by over an order of magnitude. <s> BIB001 </s> A Survey on Soft Subspace Clustering <s> Scalable Learning for Large Dataset <s> Reduced set density estimator (RSDE) is an important technique that can be used to replace the classical Parzen window estimator (PW) for saving the computational cost. Though RSDE demonstrates a nicer performance in the density accuracy and the computational time compared with several existing methods, it still faces the critical challenge for practical applications because of its high time complexity (no less than O(N^2)) and space complexity (O(N^2)) in training the model weighting coefficients on large data sets. In order to overcome this shortcoming, a fast reduced set density estimator algorithm (FRSDE) is proposed in this study. First, the relationship between RSDE and the minimal enclosing ball problems (MEB) in computational geometry is revealed. Then, the finding that RSDE is equivalent to a special MEB problem is derived. With this finding, the fast core-set based MEB approximation algorithm is introduced to develop the proposed algorithm FRSDE. Compared with RSDE, FRSDE has the following distinctive advantage: it can guarantee that the upper bound of the time complexity is linear with the size N of a large data set and the upper bound of the space complexity is independent of N. Our experimental results show that the proposed FRSDE has a competitive performance in the density accuracy and an overwhelming advantage over RSDE for large data sets in the data condensation rate and the training time for the weighting coefficients. <s> BIB002 </s> A Survey on Soft Subspace Clustering <s> Scalable Learning for Large Dataset <s> While fuzzy inference systems (FISs) have been extensively studied in the past decades, the minimum enclosing ball (MEB) problem was recently introduced to develop fast and scalable methods in pattern classification and machine learning. In this paper, the relationship between these two apparently different data modeling techniques is explored. First, based on the reduced-set density estimator, a bridge between the MEB problem and the FIS is established. Then, an important finding that the Mamdani-Larsen FIS (ML-FIS) can be translated into a special kernelized MEB problem, i.e., a center-constrained MEB problem under some conditions, is revealed. Thus, fast kernelized MEB approximation algorithms can be adopted to construct ML-FIS in an efficient manner. Here, we propose the use of a core vector machine (CVM), which is a fast kernelized MEB approximation algorithm for support vector machine (SVM) training, to accomplish this task. The proposed fast ML-FIS training algorithm has the following merits: (1) the number of fuzzy rules can be automatically determined by the CVM training and (2) fast ML-FIS training on large datasets can be achieved as the upper bound on the time complexity of learning the parameters in ML-FIS is linear with the dataset size N and the upper bound on the corresponding space complexity is theoretically independent of N. Our experiments on simulated and real datasets confirm these advantages of the proposed training method, and demonstrate its superior robustness as well. This paper not only represents a very first study of the relationship between MEB and FIS, but it also points out the mutual transformation between kernel methods and FISs under the framework of the Gaussian mixture model and MEB. <s> BIB003 </s> A Survey on Soft Subspace Clustering <s> Scalable Learning for Large Dataset <s> In order to overcome the difficulty in Takagi-Sugeno-Kang (TSK) fuzzy modeling for large datasets, scalable TSK (STSK) fuzzy-model training is investigated in this study based on the core-set-based minimal-enclosing-ball (MEB) approximation technique. The specified L2-norm penalty-based -insensitive criterion is first proposed for TSK-model training, and it is found that such TSK fuzzy-model training can be equivalently expressed as a center-constrained MEB problem. With this finding, an STSK fuzzy-model-training algorithm, which is called STSK, for large or very large datasets is then proposed by using the core-set-based MEB-approximation technique. The proposed algorithm has two distinctive advantages over classical TSK fuzzy-model training algorithms: The maximum space complexity for training is not reliant on the size of the training dataset, and the maximum time complexity for training is linear with the size of the training dataset, as confirmed by extensive experiments on both synthetic and real-world regression datasets. <s> BIB004
Another challenge to SSC is the fast and scalable learning abilities for the large datasets. As reviewed for the existing SSC algorithms, most of them are ineffective for clustering the large datasets due to high computational complexity. This problem is hampering further extensive applications of SSC. In order to overcome this challenge, some existing techniques about the scalable and fast learning on large datasets in classical clustering algorithm, can be integrated into the SSC to design the effective algorithms. Related work can be seen in  . Some new advancement for the learning on the large datasets, such as the minimal enclosing ball approximation technique  BIB001 BIB002 BIB003 BIB004 , can also be taken into account to develop the scalable and fast SSC algorithms.