Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> The self-organizing map (SOM) network was originally designed for solving problems that involve tasks such as clustering, visualization, and abstraction. While Kohonen's SOM networks have been successfully applied as a classification tool to various problem domains, their potential as a robust substitute for clustering and visualization analysis remains relatively unresearched. We believe the inadequacy of attention in the research and application of using SOM networks as a clustering method is due to its lack of procedures to generate groupings from the SOM output. In this paper, we extend the original Kohonen SOM network to include a contiguity-constrained clustering method to perform clustering based on the output map generated by the network. We compare the result with that of the other clustering tools using a classic problem from the domain of group technology. The result shows that the combination of SOM and the contiguity-constrained clustering method produce clustering results that are comparable with that of the other clustering methods. We further test the applicability of the method with two widely referenced machine-learning cases and compare the results with that of several popular statistical clustering methods. <s> BIB001 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> The growing self-organizing map (GSOM) algorithm is presented in detail and the effect of a spread factor, which can be used to measure and control the spread of the GSOM, is investigated. The spread factor is independent of the dimensionality of the data and as such can be used as a controlling measure for generating maps with different dimensionality, which can then be compared and analyzed with better accuracy. The spread factor is also presented as a method of achieving hierarchical clustering of a data set with the GSOM. Such hierarchical clustering allows the data analyst to identify significant and interesting clusters at a higher level of the hierarchy, and continue with finer clustering of the interesting clusters only. Therefore, only a small map is created in the beginning with a low spread factor, which can be generated for even a very large data set. Further analysis is conducted on selected sections of the data and of smaller volume. Therefore, this method facilitates the analysis of even very large data sets. <s> BIB002 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> The k-means algorithm with cosine similarity, also known as the spherical k-means algorithm, is a popular method for clustering document collections. However spherical k-means can often yield qualitatively poor results, especially when cluster sizes are small, say 25-30 documents per cluster, where it tends to get stuck at a local maximum far away from the optimal solution. In this paper, we present a local search procedure, which we call 'first-variation" that refines a given clustering by incrementally moving data points between clusters, thus achieving a higher objective function value. An enhancement of first variation allows a chain of such moves in a Kernighan-Lin fashion and leads to a better local maximum. Combining the enhanced first-variation with spherical k-means yields a powerful "ping-pong" strategy that often qualitatively improves k-means clustering and is computationally efficient. We present several experimental results to highlight the improvement achieved by our proposed algorithm in clustering high-dimensional and sparse text data. <s> BIB003 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> The self-organizing map (SOM) is a very popular unsupervised neural-network model for the analysis of high-dimensional input data as in data mining applications. However, at least two limitations have to be noted, which are related to the static architecture of this model as well as to the limited capabilities for the representation of hierarchical relations of the data. With our novel growing hierarchical SOM (GHSOM) we address both limitations. The GHSOM is an artificial neural-network model with hierarchical architecture composed of independent growing SOMs. The motivation was to provide a model that adapts its architecture during its unsupervised training process according to the particular requirements of the input data. Furthermore, by providing a global orientation of the independently growing maps in the individual layers of the hierarchy, navigation across branches is facilitated. The benefits of this novel neural network are a problem-dependent architecture and the intuitive representation of hierarchical relations in the data. This is especially appealing in explorative data mining applications, allowing the inherent structure of the data to unfold in a highly intuitive fashion. <s> BIB004 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> A novel extension to Kohonen's self-organising map, called the plastic self organising map (PSOM), is presented. PSOM is unlike any other network because it only has one phase of operation. The PSOM does not go through a training cycle before testing, like the SOM does and its variants. Each pattern is thus treated identically for all time. The algorithm uses a graph structure to represent data and can add or remove neurons to learn dynamic nonstationary pattern sets. The network is tested on a real world radar application and an artificial nonstationary problem. <s> BIB005 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> Clustering by document concepts is a powerful way of retrieving information from a large number of documents. This task in general does not make any assumption on the data distribution. For this task we propose a new competitive self-organising (SOM) model, namely the dynamic adaptive self-organising hybrid model (DASH). The features of DASH are a dynamic structure, hierarchical clustering, nonstationary data learning and parameter self-adjustment. All features are data-oriented: DASH adjusts its behaviour not only by modifying its parameters but also by an adaptive structure. The hierarchical growing architecture is a useful facility for such a competitive neural model which is designed for text clustering. We have presented a new type of self-organising dynamic growing neural network which can deal with the nonuniform data distribution and the nonstationary data sets and represent the inner data structure by a hierarchical view. <s> BIB006 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> In this paper, we propose a new clustering algorithm for a mixture of Gaussian-based neural network and self-growing probabilistic decision-based neural networks (SPDNN). The proposed self-growing cluster learning (SGCL) algorithm is able to find the natural number of prototypes based on a self-growing validity measure, Bayesian information criterion (BIC). The learning process starts from a single prototype randomly initialized in the feature space and grows adaptively during the learning process until most appropriate number of prototypes are found. We have conducted numerical and real-world experiments to demonstrate the effectiveness of the SGCL algorithm. In the results of using SGCL to train the SPDNN for data clustering and speaker identification problems, we have observed a noticeable improvement among various model-based or vector quantization-based classification schemes. <s> BIB007 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> We propose a novel robust clustering algorithm within the Growing Neural Gas (GNG) framework, called Robust Growing Neural Gas (RGNG) network. By incorporating several robust strategies, such as outlier resistant scheme, adaptive modulation of learning rates and cluster repulsion method into the traditional GNG framework, the proposed RGNG network possesses better robustness properties. The RGNG is insensitive to initialization, input sequence ordering and the presence of outliers. Furthermore, the RGNG network can automatically determine the optimal number of clusters by seeking the extreme value of the Minimum Description Length (MDL) measure during network growing process. The resulting center positions of the optimal number of clusters represented by prototype vectors are close to the actual ones irrespective of the existence of outliers. Topology relationships among these prototypes can also be established. Experimental results have shown the superior performance of our proposed method over the original GNG incorporating MDL method, called GNG-M, in static data clustering tasks on both artificial and UCI data sets. <s> BIB008 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> Data analysis tools are still very much a collection of data analysis methods that require analysis experts as users. On the other hand, many business users are keen to apply data analysis to business data in order to understand it or to make predictions to improve on their business decisions. In order to make state-of-the-art data analysis techniques available to such non-experts, we developed a wizard for our data analysis tool SPIDA that selects appropriate data analysis methods given soft high-level requirements. The wizard also configures and runs the chosen methods automatically. This paper describes our general approach to automating data analysis, in particular, how to select an appropriate data analysis algorithm. <s> BIB009 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> Feature selection has been widely applied in text categorization and clustering. Compared to unsupervised selection, supervised feature selection is more successful in filtering out noise in most cases. However, due to a lack of label information, clustering can hardly exploit supervised selection. Some studies have proposed to solve this problem by "pseudoclass." As empirical results show, this method is sensitive to selection criteria and data sets. In this paper, we propose a novel feature coselection for Web document clustering, which is called multitype features coselection for clustering (MFCC). MFCC uses intermediate clustering results in one type of feature space to help the selection in other types of feature spaces. Our experiments show that for most selection criteria, MFCC reduces effectively the noise introduced by "pseudoclass," and further improves clustering performance. <s> BIB010 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> The self-organizing map (SOM) neural network, also called Kohonen neural network, is an effective tool for analysis of multidimensional data. This network can be used for cluster analysis while preserving data structure (topology) in such a way that similar inputs (data) remain close together in the output layer of the network. However, no algorithm that can automatically cluster discrete groups of data is presented, and our simulation results show that the classic SOM algorithm cannot cluster discrete data correctly. In this paper, we present a novel SOM-based algorithm that can automatically cluster discrete groups of data using an unsupervised method. This method consists of three phases: at the first phase, an algorithm called ''second winner'' is performed, in which neurons in the competitive layer of the network find their initial location in the network space. At the second phase, a method called ''batch learning'' is employed, and at the end of this phase, training of the SOM network is finished. And finally at the third phase, data clustering is completed by removing the wrong links between neurons. Three real world data sets and an example of synthetic data are utilized to illustrate the accurateness and effectiveness of the proposed approach. <s> BIB011 </s> Application of Self-Organizing Maps in Text Clustering: A Review <s> Dynamic clustering of SOM <s> As recent Internet threats are evolving more rapidly than ever before, one of the major challenges in designing an intrusion detection system is to provide early and accurate detection of emerging threats. In this study, a novel framework is developed for fully unsupervised training and online anomaly detection. The framework is designed so that an initial model is constructed and then it gradually evolves according to the current state of online data without any human intervention. In the framework, a self-organizing map (SOM) that is seamlessly combined with K-means clustering is transformed into an adaptive and dynamic algorithm suitable for real-time processing. The performance of the proposed approach is evaluated through experiments using the well-known KDD Cup 1999 data set and further experiments using the honeypot data recently collected from Kyoto University. It is shown that the proposed approach can significantly increase the detection rate while the false alarm rate remains low. In particular, it is capable of detecting new types of attacks at the earliest possible time. <s> BIB012
Self-Organizing-Mapping (abbreviated as SOM) is one of the most extensively applied clustering algorithm for data analysis, because of its characteristic that its neuron topology is identical with the distribution of input data. However, the inconvenience, that it needs to predefine two parameters of cluster quantity and neuron topology, prevents it from prevailing in online situation. As indicated by Ref. BIB009 [43] BIB010 , many methods have been proposed to cluster dynamic data. For example, Dhillon et al. BIB003 proposed a dynamic clustering algorithm to help analyze the transfer of information. Unfortunately, this algorithm is time-consuming and impractical, since it needs to run several times. Ghaseminezhad and Karami BIB011 improve this algorithm by employing SOM structure, which forms an initial neuron topology at first and then dynamically tunes its topology once input data are updated. However, its neuron topology is fixed in advance and too rigid to be altered. In order to enable neuron topology easily to be altered, some self-adaptive algorithms have been proposed. The prominent merit of them is that they don't need to set any assumption about neuron topology in advance. For example, Melody in Ref. BIB001 initializes a neuron topology of small scale at first and then gradually expands it following the update of input data. Tseng et al in Ref. BIB007 improve this algorithm by tuning neuron topology in virtue of dynamically creating and deleting the arcs between different neurons. Unfortunately, aforementioned self-adaptive algorithms have two defects. One is that, when neuron topology isn't suitable for current input data, they will insert or split neurons, whereas, these newly created neurons may locate out of the area where input data distribute. The other is that, they fail to preserve topology order. Therefore, they can't perform competitive learning as transitional SOM based algorithms, which will generate some dead neurons and they will never be tuned. The detailed discussions are indicated in Ref.  BIB012 . For avoiding predefining cluster quantity, some scalable SOM based clustering algorithms are proposed, such as GSOM in Ref. BIB002 and GHSOM in Ref. BIB004 . Nevertheless, neuron topologies of them are fixed as liner, cycle, square or rectangle in advance. These kinds of topologies are too rigid, and hardly to be altered. In order to solve this problem, some topology adaptive algorithms have been proposed, such as GNG in Ref. BIB008 , PSOM in Ref. BIB005 , and DASH in Ref. BIB006 . These algorithms free of predefining neuron topology and can automatically construct it to let it conform to the distribution of input data.