TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> Support Vector Machines (SVMs) and related kernel methods have become increasingly popular tools for data mining tasks such as classification, regression, and novelty detection. The goal of this tutorial is to provide an intuitive explanation of SVMs from a geometric perspective. The classification problem is used to investigate the basic concepts behind SVMs and to examine their strengths and weaknesses from a data mining perspective. While this overview is not comprehensive, it does provide resources for those interested in further exploring SVMs. <s> BIB001 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> The primary goal of pattern recognition is supervised or unsupervised classification. Among the various frameworks in which pattern recognition has been traditionally formulated, the statistical approach has been most intensively studied and used in practice. More recently, neural network techniques and methods imported from statistical learning theory have been receiving increasing attention. The design of a recognition system requires careful attention to the following issues: definition of pattern classes, sensing environment, pattern representation, feature extraction and selection, cluster analysis, classifier design and learning, selection of training and test samples, and performance evaluation. In spite of almost 50 years of research and development in this field, the general problem of recognizing complex patterns with arbitrary orientation, location, and scale remains unsolved. New and emerging applications, such as data mining, web searching, retrieval of multimedia data, face recognition, and cursive handwriting recognition, require robust and efficient pattern recognition techniques. The objective of this review paper is to summarize and compare some of the well-known methods used in various stages of a pattern recognition system and identify research topics and applications which are at the forefront of this exciting and challenging field. <s> BIB002 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> Driven by the progress in the field of single-trial analysis of EEG, there is a growing interest in brain computer interfaces (BCIs), i.e., systems that enable human subjects to control a computer only by means of their brain signals. In a pseudo-online simulation our BCI detects upcoming finger movements in a natural keyboard typing condition and predicts their laterality. This can be done on average 100-230ms before the respective key is actually pressed, i.e., long before the onset of EMG. Our approach is appealing for its short response time and high classification accuracy (>96%) in a binary decision where no human training is involved. We compare discriminative classifiers like Support Vector Machines (SVMs) and different variants of Fisher Discriminant that possess favorable regularization properties for dealing with high noise cases (inter-trial variablity). <s> BIB003 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> The reliable operation of brain-computer interfaces (BCIs) based on spontaneous electroencephalogram (EEG) signals requires accurate classification of multichannel EEG. The design of EEG representations and classifiers for BCI are open research questions whose difficulty stems from the need to extract complex spatial and temporal patterns from noisy multidimensional time series obtained from EEG measurements. The high-dimensional and noisy nature of EEG may limit the advantage of nonlinear classification methods over linear ones. This paper reports the results of a linear (linear discriminant analysis) and two nonlinear classifiers (neural networks and support vector machines) applied to the classification of spontaneous EEG during five mental tasks, showing that nonlinear classifiers produce only slightly better classification results. An approach to feature selection based on genetic algorithms is also presented with preliminary results of application to EEG during finger movement. <s> BIB004 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> High rate classification of imagery tasks is still one of the hot topics among the brain computer interface (BCI) groups. In order to improve this rate, a new approach based on fractal dimension as feature and Adaboost as classifier is presented for five subjects in this paper. To have a comparison, features such as band power, Hjorth parameters along with LDA classifier have been taken into account. Fractal dimension as a feature with Adaboost and LDA can be considered as alternative combinations for BCI applications. <s> BIB005 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> This paper addresses the problem of signal responses variability within a single subject in P300 speller Brain-Computer Interfaces. We propose here a method to cope with these variabilities by considering a single learner for each acquisition session. Each learner consists of a channel selection procedure and a classifier. Our algorithm has been benchmarked with the data and the results of the BCI 2003 competition dataset and we clearly show that our approach yields to state-of-the art results. <s> BIB006 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> To determine and compare the performance of different classifiers applied to four-class EEG data is the goal of this communication. The EEG data were recorded with 60 electrodes from five subjects performing four different motor-imagery tasks. The EEG signal was modeled by an adaptive autoregressive (AAR) process whose parameters were extracted by Kalman filtering. By these AAR parameters four classifiers were obtained, namely minimum distance analysis (MDA)--for single-channel analysis, and linear discriminant analysis (LDA), k-nearest-neighbor (kNN) classifiers as well as support vector machine (SVM) classifiers for multi-channel analysis. The performance of all four classifiers was quantified and evaluated by Cohen's kappa coefficient, an advantageous measure we introduced here to BCI research for the first time. The single-channel results gave rise to topographic maps that revealed the channels with the highest level of separability between classes for each subject. Our results of the multi-channel analysis indicate SVM as the most successful classifier, whereas kNN performed worst. <s> BIB007 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> Gradient boosting is a machine learning method, that builds one strong classifier from many weak classifiers. In this work, an algorithm based on gradient boosting is presented, that detects event-related potentials in single electroencephalogram (EEG) trials. The algorithm is used to detect the P300 in the human EEG and to build a brain-computer interface (BCI), specifically a spelling device. Important features of the method described here are its high classification accuracy and its conceptual simplicity. The algorithm was tested with datasets recorded in our lab and one benchmark dataset from the BCI Competition 2003. The number of correctly inferred symbols with the P300 speller paradigm varied between 90% and 100%. In particular, all of the inferred symbols were correct for the BCI competition dataset <s> BIB008 </s> TOPICAL REVIEW A Review of Classification Algorithms for EEG-based Brain-Computer Interfaces <s> inria-00134950, version 1 -6 Mar 2007 <s> In recent years, brain-computer interface (BCI) technology has emerged very rapidly. Brain-computer interfaces (BCIs) bring us a new communication interface technology which can translate brain activities into control signals of devices like computers, robots. The preprocessing of electroencephalographic (EEG) signal and translation algorithms play an important role in EEG-based BCIs. In this study, we employed an independent component analysis (ICA)-based preprocessing method and a committee machine-based translation algorithm for the offline analysis of a cursor control experiment. The results show that ICA is an efficient preprocessing method and the committee machine is a good choice for translation algorithm. <s> BIB009
Generative-discriminative: Generative (also known as informative) classifiers, e.g., Bayes quadratic, learn the class models.  . Formally, classification consists in finding the true label y * of a feature vector x using a mapping f . This mapping is learnt from a training set T . The best mapping f * that has generated the labels is, of course, unknown. If we consider the Mean Square Error (MSE), classification errors can be decomposed in three terms   : These three terms describe three possible sources of classification error: • Noise: represents the noise within the system. This is an irreducible error; • Bias: represents the divergence between the estimated mapping and the best mapping. Therefore, it depends on the method that has been chosen to obtain f (linear, quadratic, . . . ); • Variance: reflects the sensitivity to the training set T used. To attain the lowest classification error, both the Bias and the Variance must be low. Unfortunatly, there is a "natural" Bias-Variance tradeoff. Actually, stable classifiers tend to have a high Bias and a low Variance, whereas unstable classifiers have a low Bias and a high Variance. This can explain why simple classifiers sometimes outperform more complex ones. Several techniques, known as stabilization techniques, can be used to reduce the Variance. Among them, we can quote combination of classifiers  and regularization (see section 2.2.1). EEG signals are known to be non-stationary. Training sets coming from different sessions are likely to be relatively different. Thus, a low Variance can be a solution to cope with the variability problem in BCI systems. kinds of linear classifier have been used for BCI design, namely, Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM). the distance from the nearest training points (see Figure 2 ). Maximizing the margins is known to increase the generalization capabilites  BIB001 . As RFLDA, an SVM uses a regularization parameter C that enables accomodation to outliers and allows errors on the training set. Such an SVM enables classification using linear decision boundaries, and is known as linear SVM. This classifier has been applied, always with success, to a relatively large number of synchronous BCI problems BIB003 [35] BIB006 . However, it is possible to create nonlinear decision boundaries, with only a low increase of the classifier's complexity, by using the "kernel trick". It consists in implicitly mapping the data to another space, generally of much higher dimensionality, using a kernel function K(x, y). The kernel generally used in BCI research is the Gaussian or Radial Basis Function (RBF) kernel: The corresponding SVM is known as Gaussian SVM or RBF SVM  [41] RBF SVM have also given very good results for BCI applications  BIB004 . As LDA, SVM has been applied to multiclass BCI problems using the OVR strategy BIB007 . SVM have several advantages. Actually, thanks to the margin maximization and the regularization term, SVM are known to have good generalization properties BIB001 BIB002 , to be insensitive to overtraining BIB002 and to the curse-of-dimensionality  BIB001 . Finally, SVM have a few hyperparameters that need to be defined by hand, namely, the regularization parameter C and the RBF width σ if using kernel 2. These advantages are gained at the expense of a low speed of execution. This section first describes the most widely used NN for BCI, which is the MultiLayer Perceptron (MLP). Then, it briefly presents other architectures of neural network used for BCI applications. The main advantage of this classifier is that one IOHMM can discriminate several classes, whereas one HMM per class is needed to achieve the same operation. the errors committed by the previous ones  . It can build up a powerful classifier out of several weak ones, and it is unlikely to overtrain. Unfortunalty, it is sensible to mislabels BIB002 which may explain why it was not succesfull in one BCI study BIB005 . To date, in the field of BCI, boosting has been experimented with MLP BIB005 and Ordinary Least Square (OLS) BIB008 ; Voting: While using Voting, several classifiers are being used, each of them assigning the input feature vector to a class. The final class will be that of the majority BIB002 . Voting is the most popular way of combining classifiers in BCI research, probably because it is simple and efficient. For instance, Voting with LVQ NN  , MLP BIB009 or SVM BIB006 have been attempted; One difficulty encountered in such a study concerns the lack of published objective comparisons between classifiers. Ideally, classifiers should be tested within the same context, i.e., with the same users, using the same feature extraction method and the same protocol. Currently, this is a crucial problem for BCI research. For this reason some researchers have proposed general purpose BCI systems such as the BCI2000 toolkit  . This toolkit is a modular framework which makes it possible to easily change the classification, preprocessing or feature extraction modules. With such a system it becomes possible to test several classifiers with the same features and preprocessings. With similar objectives of modularity, the Open-ViBE platform  proposes a framework to experiment BCI on various protocols using, for instance, neuro-feedback and Virtual Reality. An extensive use of such platforms could lead to interesting findings in the future. Table A4 . Accuracy of classifiers in pure motor imagery based BCI: multiclass and/or asynchronous case. The classes are (c1) left imagined hand movements, (c2) right imagined hand movements, (c3) imagined foot movements, (c4) imagined tongue movements, (c5) relax (baseline).