23064856
Equivalent Relationship of Feedforward Neural Networks and Real-Time Face Detection System
In this paper, we mainly investigate a fast algorithm, Extreme Learning Machine (ELM), on its equivalent relationship, approximation capability and real-time face detection application. Firstly, an equivalent relationship is presented for neural networks without orthonormalization (ELM) and orthonormal neural networks. Secondly, based on the equivalent relationship and the universal approximation of orthonormal neural networks, we successfully prove that neural networks with ELM have the property of universal approximation, and adjustable parameters of hidden neurons and orthonormal transformation are not necessary. Finally, based on the fast learning characteristic of ELM, we successfully combine ELM with AdaBoost algorithm of Viola-Jones in face detection applications such that the whole system not only retains a real-time learning speed, but also possesses high face detection accuracy.
6477031
Universal approximation using incremental constructive feedforward networks with random hidden nodes
According to conventional neural network theories, single-hidden-layer feedforward networks (SLFNs) with additive or radial basis function (RBF) hidden nodes are universal approximators when all the parameters of the networks are allowed adjustable. However, as observed in most neural network implementations, tuning all the parameters of the networks may cause learning complicated and inefficient, and it may be difficult to train networks with nondifferential activation functions such as threshold networks. Unlike conventional neural network theories, this paper proves in an incremental constructive method that in order to let SLFNs work as universal approximators, one may simply randomly choose hidden nodes and then only need to adjust the output weights linking the hidden layer and the output layer. In such SLFNs implementations, the activation functions for additive nodes can be any bounded nonconstant piecewise continuous functions g : R --> R and the activation functions for RBF nodes can be any integrable piecewise continuous functions g : R --> R and integral of R g(x)dx not equal to 0. The proposed incremental method is efficient not only for SFLNs with continuous (including nondifferentiable) activation functions but also for SLFNs with piecewise continuous (such as threshold) activation functions. Compared to other popular methods such a new network is fully automatic and users need not intervene the learning process by manually tuning control parameters.
15196694
End-to-end diagnosis of QoS violations with neural network
In this paper, we introduce a novel end-to-end approach to QoS management with respect to the diagnosis of QoS violations. We first use a set of end-to-end flow traffic statistics to describe a QoS violation. Subsequently, neural network techniques are engaged to identify and differentiate QoS violations through classification of the collected statistics. Through experiments, we find that our scheme outperforms traditional rule-based methods which require clear margins of QoS parameters in asserting a QoS violation.
31409147
Systemical convergence rate analysis of convex incremental feedforward neural networks
In this paper, we systemically investigate several convex incremental feedforward neural networks. Firstly, we prove the universal approximation and the convergence rate of a generalized convex incremental (GCI) structure, which provides us a wider parameter selection. Second, according to the convergence rate proof of GCI, we further prove the convergence rate of a best convex incremental (BCI) structure, moreover its proof also illustrates that BCI can achieve a better generalization performance than GCI. But we should note that the hidden neurons of BCI and GCI both are constructed on the maximum principle (not random). Next, we introduce the random neuron conception based on CI-ELM (convex incremental extreme learning machines), and further propose an alternative algorithm (improved CI-ELM, ICI-ELM) between CI-ELM and BCI, which removes the ''useless'' neurons in CI-ELM and improves the efficiency of neural networks. ICI-ELM randomly generates a group of parameters, among which we determine the best parameters leading to the smallest residual error. Therefore ICI-ELM can achieve a faster convergence rate than CI-ELM, meanwhile it still retains the same convergence rate as BCI. On the other hand, ICI-ELM also provides an alternative scheme to replace conventional gradient methods, which are only suitable for differential functions and often achieves local minima. The experimental results based on several benchmark regression problems support our claims.
2677048
Convergence analysis of convex incremental neural networks
Recently, a convex incremental algorithm (CI-ELM) has been proposed in Huang and Chen (Neurocomputing 70:3056–3062, 2007), which randomly chooses hidden neurons and then analytically determines the output weights connecting with the hidden layer and the output layer. Though hidden neurons are generated randomly, the network constructed by CI-ELM is still based on the principle of universal approximation. The random approximation theory breaks through the limitation of most conventional theories, eliminating the need for tuning hidden neurons. However, due to the random characteristic, some of the neurons contribute little to decrease the residual error, which eventually increase the complexity and computation of neural networks. Thus, CI-ELM cannot precisely give out its convergence rate. Based on Lee’s results (Lee et al., IEEE Trans Inf Theory 42(6):2118–2132, 1996), we first show the convergence rate of a maximum CI-ELM, and then systematically analyze the convergence rate of an enhanced CI-ELM. Different from CI-ELM, the hidden neurons of the two algorithms are chosen by following the maximum or optimality principle under the same structure as CI-ELM. Further, the proof process also demonstrates that our algorithms achieve smaller residual errors than CI-ELM. Since the proposed neural networks remove these “useless” neurons, they improve the efficiency of neural networks. The experimental results on benchmark regression problems will support our conclusions.
2891159
Convex incremental extreme learning machine
Unlike the conventional neural network theories and implementations, Huang et al. [Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Transactions on Neural Networks 17(4) (2006) 879-892] have recently proposed a new theory to show that single-hidden-layer feedforward networks (SLFNs) with randomly generated additive or radial basis function (RBF) hidden nodes (according to any continuous sampling distribution) can work as universal approximators and the resulting incremental extreme learning machine (I-ELM) outperforms many popular learning algorithms. I-ELM randomly generates the hidden nodes and analytically calculates the output weights of SLFNs, however, I-ELM does not recalculate the output weights of all the existing nodes when a new node is added. This paper shows that while retaining the same simplicity, the convergence rate of I-ELM can be further improved by recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added. Furthermore, we show that given a type of piecewise continuous computational hidden nodes (possibly not neural alike nodes), if SLFNs f"n(x)=@?i=1n@b"iG(x,a"i,b"i) can work as universal approximators with adjustable hidden node parameters, from a function approximation point of view the hidden node parameters of such ''generalized'' SLFNs (including sigmoid networks, RBF networks, trigonometric networks, threshold networks, fuzzy inference systems, fully complex neural networks, high-order networks, ridge polynomial networks, wavelet networks, etc.) can actually be randomly generated according to any continuous sampling distribution. In theory, the parameters of these SLFNs can be analytically determined by ELM instead of being tuned.
17679131
Efficient Methods on Predictions for Similarity Search over Stream Time Series
Due to the wide usage of stream time series, an efficient and effective similarity search over stream data becomes essential for many applications. Although many approaches have been proposed for searching through archived data, because of the unique characteristics of the stream, for example, data are frequently updated, traditional methods may not work for the stream time series. Especially, for the cases where the arrival of data is often delayed for various reasons, for example, the communication congestion or batch processing and so on, queries on such incomplete time series or even future time series may result in inaccuracy. Therefore, in this paper we propose two approaches, polynomial and probabilistic, to predict the unknown values that have not arrived at the system. We also present efficient indexes, that is, a multidimensional hash index and B+-tree, to facilitate the prediction and similarity search on future time series, respectively. Extensive experiments demonstrate the efficiency and effectiveness of our methods in terms of I/O, prediction and query accuracy
14544468
Using Exact Locality Sensitive Mapping to Group and Detect Audio-Based Cover Songs
Cover song detection is becoming a very hot research topic when plentiful personal music recordings or performance are released on the Internet. A nice cover song recognizer helps us group and detect cover songs to improve the searching experience. The traditional detection is to match two musical audio sequences by exhaustive pairwise comparisons. Different from the existing work, our aim is to generate a group of concatenated feature sets based on regression modeling and arrange them by indexing-based approximate techniques to avoid complicated audio sequence comparisons. We mainly focus on using exact locality sensitive mapping (ELSM) to join the concatenated feature sets and soft hash values. Similarity-invariance among audio sequence comparison is applied to define an optimal combination of several audio features. Soft hash values are pre-calculated to help locate searching range more accurately. Furthermore, we implement our algorithms in analyzing the real audio cover songs and grouping and detecting a batch of relevant cover songs embedded in large audio datasets.
1926702
Nonthreshold-Based Event Detection for 3D Environment Monitoring in Sensor Networks
Event detection is a crucial task for wireless sensor network applications, especially environment monitoring. Existing approaches for event detection are mainly based on some predefined threshold values, and thus are often inaccurate and incapable of capturing complex events. For example, in coal mine monitoring scenarios, gas leakage or water osmosis can hardly be described by the overrun of specified attribute thresholds, but some complex pattern in the full-scale view of the environmental data. To address this issue, we propose a non-threshold based approach for the real 3D sensor monitoring environment. We employ energy-efficient methods to collect a time series of data maps from the sensor network and detect complex events through matching the gathered data to spatio-temporal data patterns. Finally, we conduct trace driven simulations to prove the efficacy and efficiency of this approach on detecting events of complex phenomena from real-life records.
39616456
Advances in Web and Network Technologies, and Information Management, APWeb/WAIM 2009 International Workshops: WCMT 2009, RTBI 2009, DBIR-ENQOIR 2009, PAIS 2009, Suzhou, China, April 2-4, 2009, Revised Selected Papers
14281356
Virtual Surrounding Face Geocasting with Guaranteed Message Delivery for Ad Hoc and Sensor Networks
Geocasting in wireless sensor networks and ad hoc networks is the delivery of a message from a source to all the nodes in a given geographical region. The objectives of a geocasting protocol are two-folds: guaranteed message delivery and low transmission cost. Most of the existing protocols do not guarantee message delivery, and those that do incur a high transmission costs. In this paper, we introduce the idea of a virtual surrounding face (VSF), and present a geocasting protocol based on VSF. By using mathematical analyses and simulation studies, we show that the proposed protocol guarantees message delivery and has a significant lower transmission cost than the existing approaches.