22510443
Universal Approximation and QoS Violation Application of Extreme Learning Machine
Neural networks have been successfully applied to many applications due to their approximation capability. However, complicated network structures and algorithms will lead to computational and time-consuming burdens. In order to satisfy demanding real-time requirements, many fast learning algorithms were explored in the past. Recently, a fast algorithm, Extreme Learning Machine (ELM) (Huang et al. 70:489–501, 2006) was proposed. Unlike conventional algorithms whose neurons need to be tuned, the input-to-hidden neurons of ELM are randomly generated. Though a large number of experimental results have shown that input-to-hidden neurons need not be tuned, there lacks a rigorous proof whether ELM possesses the universal approximation capability. In this paper, based on the universal approximation property of an orthonormal method, we firstly illustrate the equivalent relationship between ELM and the orthonormal method, and further prove that neural networks with ELM are also universal approximations. We also successfully apply ELM to the identification of QoS violation in the multimedia transmission.
2891159
Convex incremental extreme learning machine
Unlike the conventional neural network theories and implementations, Huang et al. [Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Transactions on Neural Networks 17(4) (2006) 879-892] have recently proposed a new theory to show that single-hidden-layer feedforward networks (SLFNs) with randomly generated additive or radial basis function (RBF) hidden nodes (according to any continuous sampling distribution) can work as universal approximators and the resulting incremental extreme learning machine (I-ELM) outperforms many popular learning algorithms. I-ELM randomly generates the hidden nodes and analytically calculates the output weights of SLFNs, however, I-ELM does not recalculate the output weights of all the existing nodes when a new node is added. This paper shows that while retaining the same simplicity, the convergence rate of I-ELM can be further improved by recalculating the output weights of the existing nodes based on a convex optimization method when a new hidden node is randomly added. Furthermore, we show that given a type of piecewise continuous computational hidden nodes (possibly not neural alike nodes), if SLFNs f"n(x)=@?i=1n@b"iG(x,a"i,b"i) can work as universal approximators with adjustable hidden node parameters, from a function approximation point of view the hidden node parameters of such ''generalized'' SLFNs (including sigmoid networks, RBF networks, trigonometric networks, threshold networks, fuzzy inference systems, fully complex neural networks, high-order networks, ridge polynomial networks, wavelet networks, etc.) can actually be randomly generated according to any continuous sampling distribution. In theory, the parameters of these SLFNs can be analytically determined by ELM instead of being tuned.
5117817
Enhanced random search based incremental extreme learning machine
Recently an incremental algorithm referred to as incremental extreme learning machine (I-ELM) was proposed by Huang et al. [G.-B. Huang, L. Chen, C.-K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Networks 17(4) (2006) 879-892], which randomly generates hidden nodes and then analytically determines the output weights. Huang et al. [G.-B. Huang, L. Chen, C.-K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Networks 17(4) (2006) 879-892] have proved in theory that although additive or RBF hidden nodes are generated randomly the network constructed by I-ELM can work as a universal approximator. During our recent study, it is found that some of the hidden nodes in such networks may play a very minor role in the network output and thus may eventually increase the network complexity. In order to avoid this issue and to obtain a more compact network architecture, this paper proposes an enhanced method for I-ELM (referred to as EI-ELM). At each learning step, several hidden nodes are randomly generated and among them the hidden node leading to the largest residual error decreasing will be added to the existing network and the output weight of the network will be calculated in a same simple way as in the original I-ELM. Generally speaking, the proposed enhanced I-ELM works for the widespread type of piecewise continuous hidden nodes.
23064856
Equivalent Relationship of Feedforward Neural Networks and Real-Time Face Detection System
In this paper, we mainly investigate a fast algorithm, Extreme Learning Machine (ELM), on its equivalent relationship, approximation capability and real-time face detection application. Firstly, an equivalent relationship is presented for neural networks without orthonormalization (ELM) and orthonormal neural networks. Secondly, based on the equivalent relationship and the universal approximation of orthonormal neural networks, we successfully prove that neural networks with ELM have the property of universal approximation, and adjustable parameters of hidden neurons and orthonormal transformation are not necessary. Finally, based on the fast learning characteristic of ELM, we successfully combine ELM with AdaBoost algorithm of Viola-Jones in face detection applications such that the whole system not only retains a real-time learning speed, but also possesses high face detection accuracy.
36015741
Feedforward Neural Networks without Orthonormalization
A blood pressure measurement system comprises a determination means for determining whether a cuff pressure is increasing for preparation purposes or decreasing for measurement purposes. The determination means includes a comparator for comparing the current cuff pressure with the last cuff pressure, and detection means for detecting whether a difference between the current cuff pressure and the last cuff pressure is smaller than a predetermined value. Another determination means is provided for detecting whether the cuff pressure is first increased to a desired level higher than systolic pressure of a person to be measured. A reset means is provided for automatically resetting the system when the cuff pressure is decreased to a predetermined level after completion of measurement.
1909444
A Weighted Moving Average-based Approach for Cleaning Sensor Data
Nowadays, wireless sensor networks have been widely used in many monitoring applications. Due to the low quality of sensors and random effects of the environments, however, it is well known that the collected sensor data are noisy. Therefore, it is very critical to clean the sensor data before using them to answer queries or conduct data analysis. Popular data cleaning approaches, such as the moving average, cannot meet the requirements of both energy efficiency and quick response time in many sensor related applications. In this paper, we propose a hybrid sensor data cleaning approach with confidence. Specifically, we propose a smart weighted moving average (WMA) algorithm that collects confidence data from sensors and computes the weighted moving average. The rationale behind the WMA algorithm is to draw more samples for a particular value that is of great importance to the moving average, and provide higher confidence weight for this value, such that this important value can be quickly reflected in the moving average. Based on our extensive simulation results, we demonstrate that, compared to the simple moving average (SMA), our WMA approach can effectively clean data and offer quick response time.
9434179
Top-k subgraph matching query in a large graph
Recently, due to its wide applications, subgraph search has attracted a lot of attention from database and data mining community. Sub-graph search is defined as follows: given a query graph Q, we report all data graphs containing Q in the database. However, there is little work about sub-graph search in a single large graph, which has been used in many applications, such as biological network and social network.
 In this paper, we address top-k sub-graph matching query problem, which is defined as follows: given a query graph Q, we locate top-k matchings of Q in a large data graph G according to a score function. The score function is defined as the sum of the pairwise similarity between a vertex in Q and its matching vertex in G. Specifically, we first design a balanced tree (that is G-Tree) to index the large data graph. Then, based on G-Tree, we propose an efficient query algorithm (that is Ranked Matching algorithm). Our extensive experiment results show that, due to efficiency of pruning strategy, given a query with up to 20 vertices, we can locate the top-100 matchings in less than 10 seconds in a large data graph with 100K vertices. Furthermore, our approach outperforms the alternative method by orders of magnitude.
2880552
Mixed Type Audio Classification with Support Vector Machine
Content-based classification of audio data is an important problem for various applications such as overall analysis of audio-visual streams, boundary detection of video story segment, extraction of speech segments from video, and content-based video retrieval. Though the classification of audio into single type such as music, speech, environmental sound and silence is well studied, classification of mixed type audio data, such as clips having speech with music as background, is still considered a difficult problem. In this paper, we present a mixed type audio classification system based on support vector machine (SVM). In order to capture characteristics of different types of audio data, besides selecting audio features, we also design four different representation formats for each feature. Our SVM-based audio classifier can classify audio data into five types: music, speech, environment sound, speech mixed with music, and music mixed with environment sound. The experimental results show that our system outperforms other classification systems using k nearest neighbor (k-NN), neural network (NN), and Naive Bayes (NB)
14544468
Using Exact Locality Sensitive Mapping to Group and Detect Audio-Based Cover Songs
Cover song detection is becoming a very hot research topic when plentiful personal music recordings or performance are released on the Internet. A nice cover song recognizer helps us group and detect cover songs to improve the searching experience. The traditional detection is to match two musical audio sequences by exhaustive pairwise comparisons. Different from the existing work, our aim is to generate a group of concatenated feature sets based on regression modeling and arrange them by indexing-based approximate techniques to avoid complicated audio sequence comparisons. We mainly focus on using exact locality sensitive mapping (ELSM) to join the concatenated feature sets and soft hash values. Similarity-invariance among audio sequence comparison is applied to define an optimal combination of several audio features. Soft hash values are pre-calculated to help locate searching range more accurately. Furthermore, we implement our algorithms in analyzing the real audio cover songs and grouping and detecting a batch of relevant cover songs embedded in large audio datasets.
38065195
Aggregation of Multiple Reinforcement Learning Algorithms
Reinforcement learning (RL) has been successfully used in many fields. With the increasing complexity of environments and tasks, it is difficult for a single learning algorithm to cope with complicated problems with high performance. This paper proposes a new multiple learning architecture, "Aggregated Multiple Reinforcement Learning System (AMRLS)", which aggregates different RL algorithms in each learning step to make more appropriate sequential decisions than those made by individual learning algorithms. This architecture was tested on a Cart-Pole system. The presented simulation results confirm our prediction and reveal that aggregation not only provides robustness and fault tolerance ability, but also produces more smooth learning curves and needs fewer learning steps than individual learning algorithms.