Research on neural networks has gained significant momentum over the past few years. A plethora of neural networks is currently being trained on available data in research as well as in industry. Because training is a resource-intensive process and training data cannot always be made available to everyone, there has been a recent trend to attempt to re-use already-trained neural networks. As such, neural networks themselves have become research data. In this paper, we present the Neural Network Ontology, an ontology to make neural networks findable, accessible, interoperable and reusable as suggested by the well-established FAIR guiding principles for scientific data management and stewardship. We created the new FAIRnets Dataset that comprises about 2,000 neural networks openly accessible on the internet and uses the Neural Network Ontology to semantically annotate and represent the neural networks. For each of the neural networks in the FAIRnets Dataset, the relevant properties according to the Neural Network Ontology such as the description and the architecture are stored. Ultimately, the FAIRnets Dataset can be queried with a set of desired properties and responds with a set of neural networks that have these properties. We provide the service FAIRnets Search which is implemented on top of a SPARQL endpoint and allows for querying, searching and finding trained neural networks annotated with the Neural Network Ontology. The service is demonstrated by a browser-based frontend to the SPARQL endpoint.
Neural networks have been applied as a machine learning method to improve ontologies, in recent years. They are used to align @cite_1 @cite_6 @cite_2 , match @cite_13 @cite_21 or map ontologies @cite_4 @cite_19 . Furthermore, ontologies were combined with neural networks to solve different problems @cite_18 @cite_15 . However, there is no standard ontology to describe neureal networks. Still, there exists an ontology which focuses on the description of weights @cite_17 but does not fulfill the Linked-Data Principles.
@cite_18 @cite_4 @cite_21 @cite_1 @cite_6 @cite_19 @cite_2 @cite_15 @cite_13 @cite_17
2798986185 2963857521 2802636049 2526782364
Neural networks, a central tool in machine learning, have demonstrated remarkable, high fidelity performance on image recognition and classification tasks. These successes evince an ability to accurately represent high dimensional functions, potentially of great use in computational and applied mathematics. That said, there are few rigorous results about the representation error and trainability of neural networks. Here we characterize both the error and the scaling of the error with the size of the network by reinterpreting the standard optimization algorithm used in machine learning applications, stochastic gradient descent, as the evolution of a particle system with interactions governed by a potential related to the objective or "loss" function used to train the network. We show that, when the number @math of parameters is large, the empirical distribution of the particles descends on a convex landscape towards a minimizer at a rate independent of @math . We establish a Law of Large Numbers and a Central Limit Theorem for the empirical distribution, which together show that the approximation error of the network universally scales as @math . Remarkably, these properties do not depend on the dimensionality of the domain of the function that we seek to represent. Our analysis also quantifies the scale and nature of the noise introduced by stochastic gradient descent and provides guidelines for the step size and batch size to use when training a neural network. We illustrate our findings on examples in which we train neural network to learn the energy function of the continuous 3-spin model on the sphere. The approximation error scales as our analysis predicts in as high a dimension as @math . Neural networks provide state-of-the-art results for most machine learning tasks. Unfortunately, neural networks are vulnerable to adversarial examples: given an input x and any target classification t, it is possible to find a new input x' that is similar to x but classified as t. This makes it difficult to apply neural networks in security-critical areas. Defensive distillation is a recently proposed approach that can take an arbitrary neural network, and increase its robustness, reducing the success rate of current attacks' ability to find adversarial examples from 95 to 0.5 .In this paper, we demonstrate that defensive distillation does not significantly increase the robustness of neural networks by introducing three new attack algorithms that are successful on both distilled and undistilled neural networks with 100 probability. Our attacks are tailored to three distance metrics used previously in the literature, and when compared to previous adversarial example generation algorithms, our attacks are often much more effective (and never worse). Furthermore, we propose using high-confidence adversarial examples in a simple transferability test we show can also be used to break defensive distillation. We hope our attacks will be used as a benchmark in future defense attempts to create neural networks that resist adversarial examples. Abstract Neural networks are applicable in many solutions for classification, prediction, control, etc. The variety of purposes is growing but with each new application the expectations are higher. We want neural networks to be more precise independently of the input data. Efficiency of the processing in a large manner depends on the training algorithm. Basically this procedure is based on the random selection of weights in which neurons connections are burdened. During training process we implement a method which involves modification of the weights to minimize the response error of the entire structure. Training continues until the minimum error value is reached — however in general the smaller it is, the time of weight modification is longer. Another problem is that training with the same set of data can cause different training times depending on the initial weight selection. To overcome arising problems we need a method that will boost the procedure and support final precision. In this article, we propose the use of multi-threading mechanism to minimize training time by rejecting unnecessary weights selection. In the mechanism we use a multi-core solution to select the best weights between all parallel trained networks. Proposed solution was tested for three types of neural networks (classic, sparking and convolutional) using sample classification problems. The results have shown positive aspects of the proposed idea: shorter training time and better efficiency in various tasks. Recently, neuron activations extracted from a pre-trained convolutional neural network (CNN) show promising performance in various visual tasks. However, due to the domain and task bias, using the features generated from the model pre-trained for image classification as image representations for instance retrieval is problematic. In this paper, we propose quartet-net learning to improve the discriminative power of CNN features for instance retrieval. The general idea is to map the features into a space where the image similarity can be better evaluated. Our network differs from the traditional Siamese-net in two ways. First, we adopt a double-margin contrastive loss with a dynamic margin tuning strategy to train the network which leads to more robust performance. Second, we introduce in the mimic learning regularization to improve the generalization ability of the network by preventing it from overfitting to the training data. Catering for the network learning, we collect a large-scale dataset, namely GeoPair, which consists of 68k matching image pairs and 63k non-matching pairs. Experiments on several standard instance retrieval datasets demonstrate the effectiveness of our method.