A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> This completely revised second edition presents an introduction to statistical pattern recognition. Pattern recognition in general covers a wide range of problems: it is applied to engineering problems, such as character readers and wave form analysis as well as to brain modeling in biology and psychology. Statistical decision and estimation, which are the main subjects of this book, are regarded as fundamental to the study of pattern recognition. This book is appropriate as a text for introductory courses in pattern recognition and as a reference book for workers in the field. Each chapter contains computer projects as well as exercises. <s> BIB001 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Work on expert systems has received extensive attention recently, prompting growing interest in a range of environments. Much has been made of the basic concept and of the rule-based system approach typically used to construct the programs. Perhaps this is a good time then to review what we know, asses the current prospects, and suggest directions appropriate for the next steps of basic research. I'd like to do that today, and propose to do it by taking you on a journey of sorts, a metaphorical trip through the State of the Art of Expert Systems. We'll wander about the landscape, ranging from the familiar territory of the Land of Accepted Wisdom, to the vast unknowns at the Frontiers of Knowledge. I guarantee we'll all return safely, so come along.... <s> BIB002 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> We have done an empirical study of the relation of the number of parameters (weights) in a feedforward net to generalization performance. Two experiments are reported. In one, we use simulated data sets with well-controlled parameters, such as the signal-to-noise ratio of continuous-valued data. In the second, we train the network on vector-quantized mel cepstra from real speech samples. In each case, we use back-propagation to train the feedforward net to discriminate in a multiple class pattern classification problem. We report the results of these studies, and show the application of cross-validation techniques to prevent overfitting. <s> BIB003 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> We address the question of when a network can be expected to generalize from m random training examples chosen from some arbitrary probability distribution, assuming that future test examples are drawn from the same distribution. Among our results are the following bounds on appropriate sample vs. network size. Assume 0 < e ≤ 1/8. We show that if m ≥ O(W/e log N/e) random examples can be loaded on a feedforward network of linear threshold functions with N nodes and W weights, so that at least a fraction 1 - e/2 of the examples are correctly classified, then one has confidence approaching certainty that the network will correctly classify a fraction 1 - e of future test examples drawn from the same distribution. Conversely, for fully-connected feedforward nets with one hidden layer, any learning algorithm using fewer than Ω(W/e) random training examples will, for some distributions of examples consistent with an appropriate weight choice, fail at least some fixed fraction of the time to find a weight choice that will correctly classify more than a 1 - e fraction of the future test examples. <s> BIB004 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Many neural network classifiers provide outputs which estimate Bayesian a posteriori probabilities. When the estimation is accurate, network outputs can be treated as probabilities and sum to one. Simple proofs show that Bayesian probabilities are estimated when desired network outputs are 1 of M (one output unity, all others zero) and a squared-error or cross-entropy cost function is used. Results of Monte Carlo simulations performed using multilayer perceptron (MLP) networks trained with backpropagation, radial basis function (RBF) networks, and high-order polynomial networks graphically demonstrate that network outputs provide good estimates of Bayesian probabilities. Estimation accuracy depends on network complexity, the amount of training data, and the degree to which training data reflect true likelihood distributions and a priori class probabilities. Interpretation of network outputs as Bayesian probabilities allows outputs from multiple networks to be combined for higher level decision making, sim... <s> BIB005 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> It has been observed in numerical simulations that a weight decay can improve generalization in a feed-forward neural network. This paper explains why. It is proven that a weight decay has two effects in a linear network. First, it suppresses any irrelevant components of the weight vector by choosing the smallest vector that solves the learning problem. Second, if the size is chosen right, a weight decay can suppress some of the effects of static noise on the targets, which improves generalization quite a lot. It is then shown how to extend these results to networks with hidden layers and non-linear units. Finally the theory is confirmed by some numerical simulations using the data from NetTalk. <s> BIB006 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> A pattern recognition system based on Neural Network Analysis, a form of artificial intelligence, was used to search DNA flow cytometry histograms for features that correlated with breast cancer patients' risk of relapse. DNA flow cytometry histograms and clinical followup information from 796 breast cancer patients were used to train a Neural Network to predict the clinical outcome of patients in a separate independent set of 794 patients. Median follow-up in this patient data base was short, 23 months. Neural Network Analysis resulted in a model that evaluated DNA flow cytometry histograms differently than conventional analysis, which categorizes the histograms by ploidy and S-phase fraction. Neural Network Analysis appeared to identify low risk and high risk subsets of patients as accurately as conventional analysis. Neural Network Analysis placed heavy emphasis on the region to the right of the diploid G2/M peak, where a subpopulation of nuclei with high DNA content is seen even in many histograms scored as diploid by conventional techniques. The number of nuclei in this region was found to be a powerful predictor of patient outcome, and multivariate analysis showed that the number of nuclei in this region and the S-phase fraction both were independently predictive of relapse. This pilot study suggests that conventional analysis (based on a mechanistic interpretation of regions in flow cytometry histograms) might be used in conjunction with and improved by pattern recognition systems or insights derived from them. © 1993 Wiley-Liss, Inc. <s> BIB007 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> A multiple classifier system is a powerful solution to difficult pattern recognition problems involving large class sets and noisy input because it allows simultaneous use of arbitrary feature descriptors and classification procedures. Decisions by the classifiers can be represented as rankings of classifiers and different instances of a problem. The rankings can be combined by methods that either reduce or rerank a given set of classes. An intersection method and union method are proposed for class set reduction. Three methods based on the highest rank, the Borda count, and logistic regression are proposed for class set reranking. These methods have been tested in applications of degraded machine-printed characters and works from large lexicons, resulting in substantial improvement in overall correctness. > <s> BIB008 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Methods are needed to assist with automating three-color flow cytometric immunophenotyping of bone marrow from leukemia patients. Described is a method in which a normal bone marrow data set is used as a template against which to compare leukemic bone marrow data sets. This template is obtained using techniques of cluster analysis and cluster editing. Leukemic cells often inappropriately express antigens and appear in a different part of the multivariate data space than normal cells. To recognize the cells exhibiting inappropriate antigen expression, an artificial cluster of "cells" is added to the normal template. The "cells" in this cluster fill the space not occupied by normal cells. Classification and regression tree (CART) analysis is used to train a classifier that can then be used to isolate the major cell types and the inappropriate expression cells in a leukemic bone marrow specimen. <s> BIB009 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Acute leukemia is one of the leading malignancies in the United States with a mortality rate strongly influenced by the phenotype. This phenotype is based on detection of cell associated antigens normally expressed during leucopoietic differentiation. In this regard, leukemia classified as lymphoid or myeloid by phenotype is also classified as a candidate for the corresponding chemotherapy protocol. Additionally, the subtype of leukemia based on the degree of differentiation and cell maturity influence prognosis, response to treatment, and median survival times. In this paper, we analyze immunophenotype flow cytometry data toward categorization of leukemia into subcategories based on lineage and differentiation antigen expression. Twenty-eight inputs (derived from the mean fluorescence intensity of up to 27 antibodies, and an additional binary input denoting the past diagnosis of leukemia) are used as input to a neural classifier to categorize a total of 170 cases into the lineage and differentiation categories of leukemia. The neural classifier consisted of a feed forward network trained using back propagation. A complexity regulation term (weight decay) was used to improve the generalization performance of the neural classifier. A training error of 0.0% and a generalization error of 10.3% was obtained for categorization based on lineage, while a training error of 0.0% and a generalization error of 10.0% was obtained for categorization based on differentiation. These results indicate that objective classification of multifaceted phenotypes in leukemia can be achieved for analyzing multiparameter data in flow cytometry and further categorization into the prognostic subtypes. <s> BIB010 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Conventional analysis of flow cytometric data requires that population identification be performed graphically after a sample has been run using two-parameter scatter plots. As more parameters are measured, the number of possible two-parameter plots increases geometrically, making data analysis increasingly cumbersome. Artificial Neural Systems (ANS), also known as neural networks, are a powerful and convenient method for overcoming this data bottleneck. ANS "learn" to make classifications using all of the measured parameters simultaneously. Mathematical models and programming expertise are not required. ANS are inherently parallel so that high processing speed can be achieved. Because ANS are nonlinear, curved class boundaries and other nonlinearities can emerge naturally. Here, we present biomedical and oceanographic data to demonstrate the useful properties of neural networks for processing and analyzing flow cytometry data. We show that ANS are equally useful for human leukocytes and marine plankton data. They can easily accommodate nonlinear variations in data, detect subtle changes in measurements, interpolate and classify cells they were not trained on, and analyze multiparameter cell data in real time. Real-time classification of a mixture of six cyanobacteria strains was achieved with an average accuracy of 98%. <s> BIB011 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Four artificial neural network paradigms (multilayer perceptron networks, learning vector quantization networks, and radial and asymmetric basis function networks) and two statistical methods (parametric statistical classification by modelling each class with Gaussian distributions, and non-parametric density estimation via the K-nearest neighbour method) were compared for their ability to identify seven freshwater and five marine phytoplankton species from flow cytometric data. Kohonen self-organizing maps were also used to examine similarities between species. Optimized networks and statistical methods performed similarly, correctly identifying between 86.8% and 90.1% of data from freshwater species, and between 81.3% and 84.1% of data from marine species. Choice of identification technique must therefore be made on the basis of other criteria. We highlight the way each method partitions the data space and thereby separates the data clusters, and discuss the relative merits of each with reference to complexity of data boundaries, training time, analysis time and behaviour when presented with 'novel' data. <s> BIB012 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Cross validation can be used to detect when overfitting starts during supervised training of a neural network; training is then stopped before convergence to avoid the overfitting ('early stopping'). The exact criterion used for cross validation based early stopping, however, is chosen in an ad-hoc fashion by most researchers or training is stopped interactively. To aid a more well-founded selection of the stopping criterion, 14 different automatic stopping criteria from three classes were evaluated empirically for their efficiency and effectiveness in 12 different classification and approximation tasks using multi-layer perceptrons with RPROP training. The experiments show that, on average, slower stopping criteria allow for small improvements in generalization (in the order of 4%), but cost about a factor of 4 longer in training time. <s> BIB013 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> A comprehensive look at learning and generalization theory. The statistical theory of learning and generalization concerns the problem of choosing desired functions on the basis of empirical data. Highly applicable to a variety of computer science and robotics fields, this book offers lucid coverage of the theory as a whole. Presenting a method for determining the necessary and sufficient conditions for consistency of learning process, the author covers function estimates from small data pools, applying these estimations to real-life problems, and much more. <s> BIB014 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Obtaining training data for constructing artificial neural networks (ANNs) to identify microbiological taxa is not always easy. Often, only small data sets with different numbers of observations per taxon are available. Here, the effect of both size of the training data set and of an imbalanced number of training patterns for different taxa is investigated using radial basis function ANNs to identify up to 60 species of marine microalgae. The best networks trained to discriminate 20, 40 and 60 species respectively gave overall percentage correct identification of 92, 84 and 77%. From 100 to 200 patterns per species was sufficient in networks trained to discriminate 20, 40 or 60 species. For 40 and 60 species data sets an imbalance in the number of training patterns per species always affected training success, the greater the imbalance the greater the effect. However, this could be largely compensated for by adjusting the networks using a posteriori probabilities, estimated as network output values. <s> BIB015 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Radial basis function artificial neural networks (ANNs) were trained to discriminate between ::: phytoplankton species based on 7 flow cytometric parameters measured on axenic cultures. ::: Comparison was made between the performance of networks restricted to using radially-symmetric ::: basis functions and networks using more general arbitrarily oriented ellipso~dal basis functions, with ::: the latter proving significantly superior in performance. ANNs trained on 62, 54 and 72 taxa identified ::: them with respectively 77, 73 and 70% overall success. As well as high success in identification, high ::: confidence of correct identification was also achieved. Misidentifications resulted from overlap of character ::: distributions. Improved overall identification success can be achieved by grouping together species ::: with similar character distributions. This can be done within genera or based on groupings indicated ::: in dendrograms constructed for the data on all species. When an ANN trained on 1 data set was ::: tested with data on cells grown under different light conditions, overall successful identification was ::: low (<20%), but when an ANN was trained on a combined data set identification success was high ::: (>?0%). Clearly it is essential to include data on cells covering the whole spectrum of biological variatlon. ::: Ways of obtaining data for training ANNs to identify phytoplankton from field samples are discussed. <s> BIB016 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> BACKGROUND ::: Analytical flow cytometry (AFC), by quantifying sometimes more than 10 optical parameters on cells at rates of approximately 10(3) cells/s, rapidly generates vast quantities of multidimensional data, which provides a considerable challenge for data analysis. We review the application of multivariate data analysis and pattern recognition techniques to flow cytometry. ::: ::: ::: METHODS ::: Approaches were divided into two broad types depending on whether the aim was identification or clustering. Multivariate statistical approaches, supervised artificial neural networks (ANNs), problems of overlapping character distributions, unbounded data sets, missing parameters, scaling up, and estimating proportions of different types of cells comprised the first category. Classic clustering methods, fuzzy clustering, and unsupervised ANNs comprised the second category. We demonstrate the state of the art by using AFC data on marine phytoplankton populations. ::: ::: ::: RESULTS AND CONCLUSIONS ::: Information held within the large quantities of data generated by AFC was tractable using ANNs, but for field studies the problem of obtaining suitable training data needs to be resolved, and coping with an almost infinite number of cell categories needs further research. <s> BIB017 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Biological identification poses many problems which are not commonly met in other areas where artificial neural networks (ANNs) are applied. One such problem is that the number of possible classes is often unbounded and it is necessary to be able to deal with novel categories (taxa) either by rejecting them or retraining to allow for them. Where it is necessary to incorporate them, retraining a neural network may be time consuming and or impossible. One way of negating this problem is to use single-category networks that can be combined and configured as required. ::: ::: This paper compares support vector machines (SVMs) with strongly partitioned traditional radial basis function (RBF) networks for the discrimination of single species of phytoplankton against a background of N other species. SVMs resulted in greater identification success than the unpartitioned, large single RBF networks (81 and 77%, respectively), though partitioned RBF ANNs performed relatively poorly (50% success) at discrimination of 61 marine phytoplankton species from flow cytometry data. Greatest success was achieved by combining the outputs of the individual networks by means of a ‘winner takes all’ strategy; with RBF ANNs identification success dramatically increased (from 16 to 50%), though there was only a modest increase with SVMs (77–81%). When SVMs trained on one data set were tested with data on cells grown under different light conditions, an overall successful identification was low (13%), but when SVMs were trained on a combined data set identification was high (>70%). Clearly, it is essential to cover the whole spectrum of biological variation in the training data set. <s> BIB018 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> From the Publisher: ::: "In many areas of mathematics, science and engineering, from computer graphics to inverse methods to signal processing it is necessary to estimate parameters, usually multidimensional, by approximation and interpolation. Radial basis functions are a modern and powerful tool which work well in very general circumstances, and so are becoming of widespread use, as the limitations of other methods, such as least squares, polynomial interpolation or wavelet-based, become apparent." This is the first book devoted to the subject and the author's aim is to give a thorough treatment from both the theoretical and practical implementation viewpoints. For example, he emphasises the many positive features of radial basis functions such as the unique solvability of the interpolation problem, the computation of interpolants, their smoothness and convergence, and provides a careful classification of the radial basis functions into types that have different convergence. A comprehensive bibliography rounds off what will prove a very valuable work. <s> BIB019 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> We describe here the application of a type of artificial neural network, the Gaussian radial basis function (RBF) network, in the identification of a large number of phytoplankton strains from their 11-dimensional flow cytometric characteristics measured by the European Optical Plankton Analyser instrument. The effect of network parameters on optimization is examined. Optimized RBF networks recognized 34 species of marine and freshwater phytoplankton with 91. 5% success overall. The relative importance of each measured parameter in discriminating these data and the behavior of RBF networks in response to data from "novel" species (species not present in the training data) were analyzed. <s> BIB020 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> The tutorial starts with an overview of the concepts of VC dimension and structural risk minimization. We then describe linear Support Vector Machines (SVMs) for separable and non-separable data, working through a non-trivial example in detail. We describe a mechanical analogy, and discuss when SVM solutions are unique and when they are global. We describe how support vector training can be practically implemented, and discuss in detail the kernel mapping technique which is used to construct SVM solutions which are nonlinear in the data. We show how Support Vector machines can have very large (even infinite) VC dimension by computing the VC dimension for homogeneous polynomial and Gaussian radial basis function kernels. While very high VC dimension would normally bode ill for generalization performance, and while at present there exists no theory which shows that good generalization performance is guaranteed for SVMs, there are several arguments which support the observed high accuracy of SVMs, which we review. Results of some experiments which were inspired by these arguments are also presented. We give numerous examples and proofs of most of the key theorems. There is new material, and I hope that the reader will find that even old material is cast in a fresh light. <s> BIB021 </s> A Survey of Flow Cytometry Data Analysis Methods <s> Supervised Techniques for Gating. <s> Bayesian classification is currently of considerable interest. It provides a strategy for eliminating the uncertainty associated with a particular choice of classifiermodel parameters, and is the optimal decision-theoretic choice under certain circumstances when there is no single “true” classifier for a given data set. Modern computing capabilities can easily support the Markov chain Monte Carlo sampling that is necessary to carry out the calculations involved, but the information available in these samples is not at present being fully utilised. We show how it can be allied to known results concerning the “reject option” in order to produce an assessment of the confidence that can be ascribed to particular classifications, and how these confidence measures can be used to compare the performances of classifiers. Incorporating these confidence measures can alter the apparent ranking of classifiers as given by straightforward success or error rates. Several possible methods for obtaining confidence assessments are described, and compared on a range of data sets using the Bayesian probabilistic nearest-neighbour classifier. <s> BIB022
Supervised techniques require training data and a training phase to learn the relationship between the events and output classes but unsupervised ones do not need this. Selection of training data that is representative of all cell populations of interest is important in training supervised techniques. Supervised techniques usually classify the input events to one of the predefined cell populations introduced to the algorithm in the training stage. Therefore, if a novel cell population exists in the data, the algorithm classifies that population as belonging to one of the predefined cell populations and not as a novel population. Two strategies can overcome this problem to some extent. (i) The first one is assigning an "unknown" class for the input patterns that are unlikely to belong to known event categories BIB009 BIB001 BIB002 . A disadvantage of this solution is that if two novel categories exist in the test data, both will be classified as unknown even though the unknown class is comprised of multiple novel classes. It is, however, possible to add another stage of processing to further investigate the unknown events to see if they consist of multiple populations. Another similar solution would be to assume that each event can belong to several classes with different membership (e.g., event one belonging to "Class 1" with 70% chance and to "Class 2" with 30% chance) or to assign decision confidence for each classified event and reject less confident classifications as outliers or unknowns BIB001 BIB022 BIB002 . Using such a strategy, Wilkins et al. BIB020 show that more than 70% of novel species were successfully identified as "unknown" while the proportion of correctly classified species decreased moderately (from 93.8% to 86.8%) compared to the case when no novel species were identified. (ii) The second approach used by Beckman et al. BIB009 suggests adding fictitious events that reside in some of the empty spaces. Input events that are close to these fictitious events are classified as unknown events rather than being classified as belonging to the populations of interest BIB009 . This approach, however, needs extensive intervention in the data space in order to generate populations that represent unwanted event types. Moreover, this task is impractical when the dimension of the data is high, as one needs to generate fictitious data points that represent different unknown categories throughout the whole data space BIB017 . Overall, supervised techniques are suitable for tasks where we know how many classes exist in the data and a choice of unknown class would exclude the events that do not belong to the classes of interest. On the other hand, unsupervised techniques are more suitable for novel class discovery tasks. In supervised learning techniques, the training set should be a good representative of the future unseen data. Therefore, reproducible FCM data is necessary. For example, if there is excessive drift in the centroids of the cell populations, many of the cells could be misclassified. Some minor amount of drift can be usually accommodated by the algorithm itself and also having training sets composed of samples measured at different times for different individuals BIB007 . One approach to overcome this problem is to normalize the data before gating. Care should be taken when using supervised techniques, as usually unequal numbers of training patterns of each class are available, and this can bias the training of the classifier towards the classes with higher number of training events. One solution that has been suggested and applied to FCM data is to take into account a posteriori probabilities and class probabilities (i.e., the proportion of each of the cell categories in the training data) BIB005 BIB017 BIB015 . During training, a supervised learning algorithm reaches a state where, given sufficient and informative data, it should be capable of predicting the correct label for unseen data. However, the algorithm may adjust itself to very specific features of the training data that have little relation to unseen data. In this process referred to as overfitting, the performance on the training examples is high while the performance on unseen data becomes worse. Roughly speaking, an algorithm that is overfit is like a botanist with a photographic memory who, when presented with a new tree, concludes that it is not a tree because it has a different number of leaves from anything he/she has seen before BIB021 . Overfitting can be avoided by employing techniques such as regularization and early stopping BIB006 BIB003  . Regularization involves introducing a form of penalty for complexity of the classification model. An example of regularization in neural networks is weight decay algorithm used in MLP neural networks. As large weights can decrease the performance of an MLP classifier on unseen data, weight decay penalizes the large weights causing the weights to converge to smaller absolute values than they otherwise would BIB006 . This strategy has been used in the context of gating FCM data BIB010 . In early stopping, the available training data is divided into two sets, that is, a new training set and a validation set. In each iteration of learning, the data of the new training set is used to train the learning algorithm and the validation set is used to evaluate its performance. The learning phase is forced to stop once the performance on the validation set does not improve or degrades. This method can be used either interactively (based on human intervention) or automatically (based on some stopping criteria usually chosen in an adhoc fashion). As mentioned in BIB013 , early stopping is widely used as it is easy to implement and has been reported to be superior to regularization methods in many cases (e.g.,  ). A number of algorithms in the category of supervised techniques such as multilayer perceptron (MLP) networks (e.g., BIB011 BIB012 ), radial basis function (RBF) networks (e.g., BIB012 BIB020 ), and support vector machines (SVM) BIB018 have been used in the context of cell population identification in FCM data. A typical MLP network consists of a set of nodes forming the input layer, one or more hidden layers, and an output layer. The MLP network has a highly connected topology since every input node is connected to all nodes in the first hidden layer, every node in the hidden layers is connected to all nodes in the next layer, and so on. The value of each node is determined by a weighted combination of input nodes, possibly including some nonlinear activation function. An MLP network is trained by repeated presentation of input patterns to the network. During the training process, small iterative weight changes in the structure of the network are performed until the predicted outputs are considered close enough to desired outputs. Designing an MLP classifier is not a trivial task as one needs to determine optimal parameters of the MLP structure (e.g., number of hidden layers, number of hidden layer nodes, etc.) for each specific classification task. For most problems, one hidden layer is sufficient. Using two hidden layers rarely improves the model, and it may introduce a greater risk of converging to a local minima. The network may not be able to model complex data if inadequate number of hidden layer nodes is used. On the other hand, if too many nodes are used, the training time may become excessively long, and the network may overfit the data. In general, training an MLP is relatively slow and sometimes the algorithm gets stuck in local minima and therefore the training process has to be restarted  . It has been shown that if an accuracy of (1 − e) on a test set is desirable, the number of events in the training set, p, should satisfy p ≥ w/e, where w is the total number of weights in the network BIB004 . Hence, to obtain 90% accuracy (e = 0.1) on test set, the desirable number of events required in training set is at least ten times the total number of weights. While having p ≥ w/e is definitely desirable, it is sometimes difficult in practice to build such a large database of clinical cases. An option is to use a perturbation method to generate a large number of cases by introducing small variations in actual cases BIB010 . The importance of having sufficiently large training sets to cover biological variation is highlighted by the increase in overall identification success of different marine microalgae in an FCM study BIB005 . An RBF neural network typically is comprised of three layers of nodes (i.e., input, hidden and output layers). The neurons in the hidden layer contain basis functions, usually Gaussian transfer functions whose outputs are inversely proportional to the distance from the center of the basis function. Normally the Euclidean distance is used as the distance measure, although other distance functions are also possible. An RBF network output is formed by a weighted sum of the hidden layer neuron outputs and the unity bias. The parameters of an RBF network which are determined in the training stage consist of the positions of the basis function centers, the radius (spread) of the basis functions in each dimension, the weights in output sum applied to the hidden layer nodes outputs as they are passed to the summation layer, the parameters of the linear part, and so forth. Various methods have been used to train RBF networks. One approach first uses k-means clustering to find cluster centers which are then used as the centers for the RBF functions. However, k-means clustering is a computationally intensive procedure, and it often does not generate the optimal number of centers. Another approach is to use a random subset of the training points as the centers. Assuming that the data is linearly separable, among the infinite number of hyperplanes that separate the data, an SVM classifier picks the one that has the smallest generalization error. Intuitively, a good choice is the hyperplane that leaves the maximum margin between the two classes, where the margin is defined as the sum of the distances of the hyperplane from the support vectors. Support vectors are the examples closest to the separating hyperplane and the aim of an SVM classifier is to orientate this hyperplane in such a way that it is as far as possible from the closest members of both classes. If the two classes are nonseparable we can still look for the hyperplane that maximizes the margin and that minimizes a quantity proportional to the number of misclassification errors. The trade-off between margin and misclassification error is controlled by a positive constant C (referred to as error penalty) that has to be chosen beforehand BIB021 BIB014 . SVMs are very universal learners. In their basic form, SVMs learn linear threshold function. Nevertheless, by a simple "plug-in" of an appropriate kernel function, they can be extended to nonlinear classifiers such as polynomial classifiers, radial basis function (RBF) networks, and threelayer sigmoid neural networks. Perhaps the biggest limitation of the SVM approach lies in the choice of the kernel. Once the kernel is fixed, SVM classifiers have only one user-chosen parameter (the error penalty) BIB021 . RBF networks can be trained significantly faster than MLPs. In addition to the number of hidden layers, a difference between RBF and MLP classifiers lies in the nodes of the hidden layer, which use different kernels (basis functions) to represent the data. RBF networks have the advantage of not suffering from local minima in the same way as MLPs. While for an RBF there is no restriction on decision boundaries formed, an MLP forms convex decision boundaries. Moreover, RBF's hidden layer performs a nonlinear mapping from the input space into a (usually) higherdimensional space in which the input patterns become linearly separable BIB019 . Although RBF networks are quick to train, when training is finished and it is being used, it is slower than an MLP. Therefore, where speed is a factor an MLP may be more appropriate. SVM can be seen as a new way to train polynomial, neural network, or RBF classifiers. While most of the techniques used to train the above mentioned classifiers are based on the idea of minimizing the training error, which is usually called empirical risk, SVMs operate on another induction principle, called structural risk minimization, which minimizes an upper bound on the generalization error BIB014 . In the context of FCM data analysis, Boddy et al. BIB016 compares the performances of RBF networks using different basis functions. Specifically, radially symmetric and a more general arbitrarily oriented ellipsoidal basis functions were employed, with the latter proving to be significantly superior in performance. The distance between input patterns and the basis function centers are defined by a distance metric, which determines the shape of the basis function. The Euclidean distance metric produces hyperspherical (radially symmetric) basis functions around the basis functions centers. Mahalanobis distance metric, on the other hand, allows the hyperellipsoid (nonradially symmetric) to adopt any orientation that best fits the data distributions. Wilkins et al. BIB012 compare several classification algorithms such as MLP, RBF, and LVQ (learning vector quantization) to identify phytoplankton species from FCM data. The authors show that identification success was more or less similar using the above-mentioned techniques. Therefore, they suggest using the criteria mentioned earlier and characteristics of the data at hand to decide which method is the best to use. In another study on phytoplankton species, Morris et al. BIB018 demonstrate that an SVM classifier outperforms RBF classification. These studies focus on specific data sets and their generalization on other data sets is unknown. Therefore, picking an algorithm based on the type of data at hand and above-mentioned characteristics of learning algorithms is recommended. One approach that might be worth considering in FCM studies is the multiple classifier systems (MCSs) BIB008 . MCSs are based on combining the outputs of ensembles of different classifiers (supervised learning techniques). Classification accuracy improvements are possible provided that a suitable combination function is designed and that the individual classifiers make different errors. Ideally, a combination function should take advantage of the strengths of individual classifiers, avoid their weaknesses, and improve classification accuracy BIB008 . (iv) able to determine the number of cell populations as the number of cell populations present in the data is usually not known apriori, (v) able to handle outliers as data can contain significant number of outliers.