Self-organizing networks such as Neural Gas, Growing Neural Gas and many others have been adopted in actual applications for both dimensionality reduction and manifold learning. Typically, the goal in these applications is obtaining a good estimate of the topology of a completely unknown subspace that can be explored only through an unordered sample of input data points. In the approach presented here, the dimension of the input manifold is assumed to be known in advance. This prior assumption can be harnessed in the design of a new, growing self-organizing network that can adapt itself in a way that, under specific conditions, will guarantee the effective and stable recovery of the exact topological structure of the input manifold.
In the Grow-When-Required (GWR) algorithm @cite_14 , which is a development of GNG, the error variable associated to each GNG unit is replaced by a firing counter @math that decreases exponentially each time the unit is winner, i.e. closest to the input signal. When @math gets below a certain threshold @math , the unit is deemed and its behavior changes. The adaptation of unit being closest to the input takes place only if the distance is below a certain threshold @math , otherwise a new unit is created. This means that the network continues to grow until the input data sample is completely included in the union of balls of radius @math centered in each unit.
@cite_14
2046372172 2531279268 1736159137 2907047316
The ability to grow extra nodes is a potentially useful facility for a self-organising neural network. A network that can add nodes into its map space can approximate the input space more accurately, and often more parsimoniously, than a network with predefined structure and size, such as the Self-Organising Map. In addition, a growing network can deal with dynamic input distributions. Most of the growing networks that have been proposed in the literature add new nodes to support the node that has accumulated the highest error during previous iterations or to support topological structures. This usually means that new nodes are added only when the number of iterations is an integer multiple of some pre-defined constant, λ.This paper suggests a way in which the learning algorithm can add nodes whenever the network in its current state does not sufficiently match the input. In this way the network grows very quickly when new data is presented, but stops growing once the network has matched the data. This is particularly important when we consider dynamic data sets, where the distribution of inputs can change to a new regime after some time.We also demonstrate the preservation of neighbourhood relations in the data by the network. The new network is compared to an existing growing network, the Growing Neural Gas (GNG), on a artificial dataset, showing how the network deals with a change in input distribution after some time. Finally, the new network is applied to several novelty detection tasks and is compared with both the GNG and an unsupervised form of the Reduced Coulomb Energy network on a robotic inspection task and with a Support Vector Machine on two benchmark novelty detection tasks. Anomaly detection is still a challenging task for video surveillance due to complex environments and unpredictable human behaviors. Most existing approaches train offline detectors using manually labeled data and predefined parameters, and are hard to model changing scenes. This paper introduces a neural network based model called online Growing Neural Gas (online GNG) to perform an unsupervised learning. Unlike a parameter-fixed GNG, our model updates learning parameters continuously, for which we propose several online neighbor-related strategies. Specific operations, namely neuron insertion, deletion, learning rate adaptation and stopping criteria selection, get upgraded to online modes. In the anomaly detection stage, the behavior patterns far away from our model are labeled as anomalous, for which far away is measured by a time-varying threshold. Experiments are implemented on three surveillance datasets, namely UMN, UCSD Ped1 Ped2 and Avenue dataset. All datasets have changing scenes due to mutable crowd density and behavior types. Anomaly detection results show that our model can adapt to the current scene rapidly and reduce false alarms while still detecting most anomalies. Quantitative comparisons with 12 recent approaches further confirm our superiority. HighlightsOnline GNG employs neighbor-related strategies to adjust dominant learning parameters in a fully automated way.Online GNG can estimate the learning efficiency and adjust the network size to fit the unstable data space dynamically.Online GNG effectively reduces the false alarms and leak detections caused by model aging which frequently happens in changing surveillance scenes. We study broadcasting time in radio networks, modeled as unit disk graphs (UDG). showed that broadcasting time depends on two parameters of the UDG network, namely, its diameter D(in hops) and its granularityg. The latter is the inverse of the densitydof the network which is the minimum Euclidean distance between any two stations. They proved that the minimum broadcasting time is @math , assuming that each node knows the density of the network and knows exactly its own position in the plane. In many situations these assumptions are unrealistic. Does removing them influence broadcasting time? The aim of this paper is to answer this question, hence we assume that density is unknown and nodes perceive their position with some unknown error margin i¾?. It turns out that this combination of missing and inaccurate information substantially changes the problem: the main new challenge becomes fast broadcasting in sparse networks (with constant density), when optimal time is O(D). Nevertheless, under our very weak scenario, we construct a broadcasting algorithm that maintains optimal time @math for all networks with at least 2 nodes, of diameter Dand granularity g, if each node perceives its position with error margin i¾?= i¾?d, for any (unknown) constant i¾?< 1 2. Rather surprisingly, the minimum time of an algorithm stopping if the source is alone, turns out to be i¾?(D+ g2). Thus, the mere stopping requirement for the special case of the lonely source causes an exponential increase in broadcasting time, for networks of any density and any small diameter. Finally, broadcasting is impossible if i¾?i¾? d 2. The advent of deep learning is a breakthrough in artificial intelligence, for which a theoretical understanding is lacking. Supervised deep learning involves the training of neural networks with a large number @math of parameters. For large enough @math , in the so-called over-parametrized regime, one can essentially fit the training data points. Sparsity-based arguments would suggest that the generalization error increases as @math grows past a certain threshold @math . Instead, empirical studies have shown that in the over-parametrized regime, generalization error keeps decreasing with @math . We resolve this paradox, through a new framework. We rely on the so-called Neural Tangent Kernel, which connects large neural nets to kernel methods, to show that the initialization causes finite-size random fluctuations @math of the neural net output function @math around its expectation @math . These affect the generalization error @math for classification: under natural assumptions, it decays to a plateau value @math in a power-law fashion @math . This description breaks down at a so-called jamming transition @math . At this threshold, we argue that @math diverges. This result leads to a plausible explanation for the cusp in test error known to occur at @math . Our results are confirmed by extensive empirical observations on the MNIST and CIFAR image datasets. Our analysis finally suggests that, given a computational envelope, it is best to use several nets of intermediate sizes, just beyond @math , and to average their outputs.