The problem of neural networks communication

In spite of the successful application of artificial neural networks (ANNs) for the solution of multiple problems (forecasting, language translation, image classification, voice recognition etc.), ANNs are still autonomous entities incapable of communication or exchange of their knowledge. Meanwhile, the ability to communicate is critical for further development of methods of artificial intelligence. We propose and test several methods of communication and knowledge fusion of ANNs. These methods do not require the presence of the initial training data and use only the internal parameters of ANNs. We propose generative iterative and non-iterative methods of ANNs communication. Noniterative methods show the classification accuracy similar to that provided by an ensemble of ANNs. The accuracy of generative methods is similar to a network trained on the joint dataset.


Introduction
The ability to communicate and exchange their knowledge is crucial for artificial neural networks (ANNs) and is necessary for further development of artificial intelligence and self-learning systems. Modern ANNs, however, behave as completely autonomous units incapable of any information exchange. Additional difficulty comes from the fact that ANNs store the acquired knowledge in a distributed way amongst the millions of internal parameters (weights). Such an implicit representation of knowledge complicates its interpretation, interferes with its update, and obstructs its exchange with other neural networks. If joint use of knowledge is needed, there exists two current approaches: 1) training new neural network with a joint dataset; 2) using an ensemble of networks to produce joint prediction [1]. However, there could be situations when both of these approaches become unacceptable: previously used training data may become unavailable; storing and maintaining an ensemble of ANNs may become too resource demanding. Ideally, neural networks should be able to exchange their knowledge without any external help; the result of this exchange would be a single ANN containing the fused knowledge (in the following we denote this as fANN).
The problem of knowledge fusion of several ANNs may be divided into two separate subproblems: extraction of knowledge from one neural network and its incorporation into another ANN. The problem of knowledge extraction is essential for understanding of the functioning of neural networks. Without this understanding, ANNs cannot be used in such critical applications as IOP Publishing doi: 10.1088/1742-6596/1368/5/052033 2 transportation and health care. The problem of new knowledge incorporation into a neural network is closely related to the problem of catastrophic interference: when trained on new piece of data without an access to previous datasets, ANN quickly forgets all the previously learned knowledge [2]. Thus, the development of algorithms for communication and knowledge exchange between neural networks would simultaneously solve other important problems of machine learning. At the same time, one can try to find methods that do not require explicit extraction and embedding of knowledge, but instead operate with internal parameters of networks.
This paper is devoted to the development of methods of knowledge fusion of neural networks that do not lead to the forgetting of previously learned content and do not require previous training data. The need in the integration of knowledge from different ANNs without an access to previous training data may appear in various situations. One example is medical expert systems, which training data could be confidential, but joint knowledge can lead to better diagnosis. Another example is autonomous vehicles, which can benefit from the experience of peers. Previously, the problem of knowledge fusion was solved only for ANNs with similar weights [3] and for Support Vector Machines [4]. We developed both noniterative methods and generative iterative approaches for the solution of knowledge fusion problem for ANNs trained independently. Noniterative methods allow fast knowledge fusion; generative methods provide more accuracy.
Let us note that the problem of knowledge fusion should not be confused with the term "knowledge transfer" (or "transfer learning") [5]. According to transfer learning concept, the weights of deep network trained on one set of data can be used for training on another dataset from a similar area to speed up the training [5]. Thus, transfer learning is more concerned with the learning of new task rather than maintaining all the knowledge. Examples of transfer learning can be found in such areas as computer vision [6], language processing [7] etc. In contrast to knowledge transfer, knowledge fusion tries to integrate all the knowledge of initial networks into a single ANN. Another difference with knowledge transfer is the absence of training data, which makes the problem of knowledge fusion even more challenging.

Problem statement
In its simplest form, the problem of knowledge fusion can be stated as follows. Two ANNs (A and B) with a similar structure are trained on a classification task by some standard method (for example, by backpropagation). Each neural network is trained on its own dataset. After the training is complete, each neural network contains different knowledge. It is necessary to modify the parameters of one of the networks (for example, by changing the weights) in such a way that the resulting network would be able to reproduce the response of both of the initial networks as if it was trained on a joint dataset (figure 1). During fusion of knowledge, we assume that all the previous training data are no longer accessible. In experiments performed in this work, we use one-hot encoding in the output layers of ANNs. This encoding assumes activation of one output neuron corresponding to a specific class. Thus, the For the fusion of knowledge of deep networks, we additionally utilize the concept of transfer learning [5]. A typical architecture of deep networks consists of a convolutional part, which extracts the features from raw data, and a fully connected classifier on top that performs the separation of the features into classes. The process of training of deep networks consisted of several stages. First, the weights of some network trained on large image dataset (for example, ImageNet [8]) were used for initialization of the weights of the convolutional part. The large set of classes of ImageNet database makes the pretrained network an ideal candidate for transfer learning. At the second stage, the bottom convolutional part was supplemented by an untrained classifier, which was later trained on various classification tasks (for example, classification of animals or urban objects). At the third stage, the convolutional part of each network underwent fine tuning leading to the final slight modification of the weights. As a result, we obtained a set of deep networks with different set of weights and containing different knowledge.
Next section describes the methods for the merging the knowledge contained in several networks in more details.

Method of weights summation
The method of weights summation (WS) is based on a probabilistic process of summation of a large number of random variables (weights of ANNs) [9]. The weights of the network with a fused knowledge F can be obtained by direct summation of weights of networks and : In spite of its simplicity, this algorithm results in a rather high classification accuracy of the final fused network. We propose the following explanation of the efficiency of WS method. For some feature vector belonging to class of network , the presynaptic activity in fANN is determined by the following expression: In principle, the trained neural networks are robust under the perturbation of the weights. Thus, if the second term at the right-hand side of equation (1) is significantly smaller that the first one, the fused network will still be able to perform correct classification. To ensure the inequality , ≫ , for every node (where ( , ), are the weights connected to the node ), several conditions should be met. First, networks and should be well trained in a sense that they produce noticeable presynaptic activity in the presence of native features and low activity in the presence of foreign ones. Second, the mean of the weights' probability distribution within one layer should be zero 〈 〉 〈 〉 0. Indeed, the weights of one layer , , , and components of the feature vector , may be considered as random variables taken from some distribution. For network , feature vector contains independent random variables. The product of independent random variables ( ) can be rewritten as where is the number of components in the feature vector and average 〈 , 〉 is taken over presynaptic weights of corresponding layer. It follows from equation (2), that the term , will be close to zero if the expected value of weights distribution is zero 〈 〉 ≈ 0. The property 〈 〉 0 is valid for common training techniques [10] and is maintained in current research.
In our experiments, the WS algorithm was tested on shallow fully-connected feed-forward networks and on deep convolutional networks. The experiments with fully-connected networks were performed on a set of handwritten digits MNIST [11]. The whole dataset consists of 60000 training images of size 28 by 28 and 10000 test samples corresponding to 10 classes. We arbitrarily divided the whole dataset into two equal sets containing 5 classes each. The intensity of every input image was normalized to be in the range [0,1]. Each network and was trained on data corresponding only to the target classes of particular network. Thus, each network never saw the samples corresponding to the classes of another network and could not meaningfully classify corresponding images. The training algorithm was implemented in Keras 2.2 with the TensorFlow backend. Both networks and had an input layer with 784 nodes corresponding to the number of pixels in input images, one hidden layer with 800 nodes, and output layer with 5 nodes corresponding to the number of classes. All hidden layers had an additional bias term. The nodes in the hidden layer had ReLU activation functions. For the output layer, we used softmax activation; the cross-entropy was used as a loss function. Before training, all the weights were initialized randomly from a normal distribution with zero mean with a standard deviation 0.05. L2 regularization was used during training with L2 0.001. Adam iterative method was used for training with all the parameters set to the Keras' default values. In each experiment, a batch size of 200 was used. Early stopping was used while testing the accuracy on a validation set consisting of 12000 samples. We quantify the efficiency of classification of each network through accuracy (percent of correctly classified images). Simulations were repeated 10 times with random initialization of weights and with random subdivision of digits into classes to accumulate proper statistics.
The accuracy of the original networks A and B on partial datasets achieved 0.983±0.002. After the fusion of knowledge by weights summation, the accuracy of fANN turned out to be 0.864±0.035. This number is certainly smaller than the accuracy of a network trained on a joint dataset 0.968±0.002, but still much larger than a chance level.
We also compared the accuracy of fANN to the classification accuracy of an ensemble of networks A and B. The ensemble prediction was chosen on a base of maximum activity among all output neurons of all networks. The accuracy of fANN appeared to be just a little lower than the accuracy of an ensemble 0.900±0.021. However, one should remember that fANN replaces the ensemble of networks with just one neural net.
Experiments with deep networks were performed on separate classes of images of ImageNet project [8] (www.image-net.org). Considering the variety of the objects indexed in the project, one can construct the networks trained to classify either similar or dissimilar classes of objects. In particular, for current experiment, two "indigenous" neural networks were trained to classify 11 African and 11 Australian animals (table 1). The third "urban" neural network was trained to classify 11 objects pertinent to urban life (table 1). Each class had approximately 1500 samples.
As in experiments with MNIST dataset, it was assumed that the original trained networks A and B observed only the objects of their own classes and never saw the objects from the classes of other networks. The aim was to embed the knowledge contained in one neural network into another one in such a way that the resulting network can recognize the objects (e.g., animals or household item) from the previously unknown category even without ever "seeing" the objects from those categories before.
For the architecture of networks and , we chose VGG-16 model [12] included in Keras version 2.2 with its weights pre-trained on 1000 ImageNet classes. Those 1000 classes include large variety of objects, but, still, do not contain all the classes used in our experiments.
With the concept of knowledge transfer, the pretrained convolutional part of VGG-16 network was a good starting point for the training on previously unknown objects. In the pretrained VGG-16 network, we kept only its bottom convolutional part that we used as universal feature extractor. On top of this convolutional part, we placed a custom classifier consisting of one hidden layer with 128 nodes and one output layer with 11 neurons corresponding to the number of classes of the networks being fused. Dropout was used for the weights connecting hidden to output nodes of the classifier. ReLU activation function was used for the hidden layer, softmax activation was used for the output layer.
The input to VGG-16 network has to be a fixed-size 224 × 224 RGB image that was cropped out of a center of input raw images. Augmentation of dataset was not used. All the images were preprocessed by scaling the intensity of every image to the range [0,1]; 15% of the images were dedicated to validation and another 15% were used for the test. For the training, we used 64 pictures batch, and the training was performed with Adam optimizer on a computer with GPU support (Ubuntu 16.04 х64, Intel core i3, 6GB RAM, Nvidia GeForce GTX 1050 Ti 4GB).
The training procedure for the initial networks A and B was performed in several steps. First, the bottom convolutional part was frozen, and the top classifier was initialized with random weights and trained with small learning rate (10 -5 ) over 4 epochs. Then we froze the weights of the classifier and unfroze the top block (3 layers) of VGG-16 convolutional part. This block was trained over 2 epochs with the same small learning rate. Such a training procedure resulted in networks and different not only in their top fully connected parts, but also in their convolutional layers. As a result, the networks trained to classify African and Australian animals achieved 0.951±0.005 and 0.933±0.004 accuracy, correspondingly. The network trained to classify urban objects achieved 0.955±0.003 accuracy. The main source of classification errors was the mixing between classes with similar features. For example, the African network often misclassified chimpanzees and gorillas. The Australian network, in its turn, misclassified kangaroos and wallabies.
As the weights of the convolutional parts of both networks A and B originated from the same set of pretrained weights, during fusion by WS method, those weights were averaged according to the approach adopted by Smith & Gashler [3]. The weights of the classifiers on the top of networks were obtained from an independent set of weights and thus were added together according to the weights summation method. Experiments demonstrated that the classification accuracy of the network with merged knowledge depends on the similarity of the features of the objects, classified by original networks and . For example, if both networks and were trained to recognize the objects of the same category, then the network with fused knowledge would confuse the objects with similar sets of features. Figure 2 shows the result of knowledge fusion between two networks in the case when they were trained to recognize African and Australian animals. fANN is confident (classification accuracy is of the order of 90%) about animals with unique features, for example, zebras, but confuses similar animals from different continents (for example, emus and ostriches). An average classification accuracy of fANN is 76%. For comparison, the classification accuracy of the same set of classes by an ensemble of networks A and B is 81%.
In the case of very different set of features of the objects recognized by networks and (for example, in the case of living and inanimate objects), the classification accuracy of fANN increases. Figure 3 shows the result of knowledge fusion from two networks in a case when one of the networks was trained to recognize African animals and the other was trained to recognize urban objects. An average classification accuracy of fANN was 81%, which is better than in previous example. The classification accuracy by an ensemble of networks was 88%.

Method of elastic weight consolidation
The second noniterative method that we used for the fusion of knowledge of ANNs was based on an idea of selective modification of weights. If one changes the weights of networks A and B until these weights coincide with additional condition that loss functions ( ) and ( ) increase insignificantly The increase of total loss function F ( F ) ( F ) + ( F ) is minimal if we change only less important weights for current task. Following paper [13], we call this method Elastic Weight Consolidation (EWC).
To estimate the change of the loss function, one can decompose it into Taylor's series with respect to weights. Here is an example of such a decomposition for network : Here are the optimal weights of network trained on corresponding dataset; the summation in equation (3) is performed over all the weights. As the number of parameters reaches millions in modern neural networks, the computation and the storage of the whole matrix of second order partial derivatives (Hessian) 2 ( ) ⁄ is impossible. Following paper [13], we approximated the whole Hessian by its diagonal part: where , ≡ 2 2 ⁄ . Expressions similar to (3) and (4) should be written for network as well. Factors , , , in equation (4) express the importance of weights , , , . Usually, one needs to calculate these coefficients in advance during the training of neural network. It is easy to show that the weights of fANN F that minimize the loss function F can be calculated with the following formula In principle, equation (5) solves the problem of fusion knowledge of two ANNs. However, weights , of independently trained networks are not necessarily located in a vicinity of one another, and decomposition (3) may become insufficient. In attempt to bring closer the values of and , we performed an "alignment" of networks by rearranging hidden neurons. The contribution to the overall loss function F from a pairing of a node of network and a node of network is To find the optimal pairing of neurons from networks A and B, one needs to solve, so called, Assignment Problem [14]. The coefficients compose the cost matrix that serves as input for the Assignment Problem algorithm.
For experiments with EWC method, we used the networks with the same architecture as in WS experiments. Similarly, the experiments were performed on shallow networks with MNIST data and on deep convolutional networks with ImageNet data. In spite of heavier computations, EWC demonstrated performance very similar to the one of weights summation method. This tells us that noniterative methods of knowledge fusion are limited in achieving high classification accuracy. An alternative approach can be developed on the base of generative models.

Generative methods
Recently, neural networks were developed that can generate data of the same type and similar probabilistic distribution ( ) as the initial training data. One of such ANNs are generativeadversarial networks (GANs) [15]. Another type of generative neural networks is variational autoencoder (VAE) [16].
The idea behind the generative knowledge fusion is in the following. The knowledge of the trained classifier is its ability to predict the probability ( | ) of an object belonging to class . If another network somehow learns to reproduce this probability than it acquires the knowledge of the first network. Moreover, original dataset { } is not necessary; one just need to reproduce probability ( |̃) for any ̃ sampled from probability ( ). Generative models can provide both samples ̃ from probability ( ) and soft targets ̃ representing probabilities of these samples to belong to certain classes. For our experiments, we used variational autoencoders [16]. The architecture of VAE (figure 4a) was similar to the one used by van de Ven & Tolias [17] for prevention of catastrophic interference in sequential learning. Regular classifier was supplemented by generative part (figure 4a). The generative part includes standard VAE components [16]: sampler from latent distribution ( , ) and decoder (figure 4a). The loss function of the resulting VAE total consisted of three parts: categorical crossentropy classification error class , binary cross-entropy reconstruction loss recon , and latent variable regularization term KL [16,17]  To prevent catastrophic forgetting, synthetic data should be generated not only from knowledge donor but also from knowledge acceptor (figure 4b).
The experiments with generative approach were performed on MNIST dataset. The classifier of the same structure as the one described in section 3.1 was supplemented by generative part (figure 4a) consisting of two layers with 8 nodes each encoding latent distribution mean and variance , one hidden layer with 800 neurons (decoder) and ReLU activation function, and output layer with sigmoid activation function and 784 nodes corresponding to the dimension of the original images.
As in previous experiments, MNIST dataset was arbitrarily divided into two parts containing 5 classes each. Networks A and B were trained over 40 epochs with Adam optimizer to properly classify and generate images of digits. The classification accuracy of networks A and B achieved 0.983±0.004. After the networks A and B have being trained, 60000 artificial samples ̃ with corresponding soft labels ̃ were generated from the decoder by supplying it with random variable sampled from standard normal distribution. The examples of generated images are shown in figure 5. As usually happens with images generated by VAEs, the digits appear somewhat blurred. The generated datasets {̃,̃}, {̃,̃} were supplied to network B (figure 4b); network B was trained on this combined dataset over 40 epochs. The accuracy of the retrained network B tested on complete MNIST test set was 0.929±0.006. This is considerably higher than the accuracy of noniterative knowledge fusion, higher than the accuracy of an ensemble of networks, and it is getting close to the accuracy of a network trained on joint dataset (0.9698±0.0006). The shortcoming of generative method however is larger network, that should incorporate generative part and longer training time.

Conclusion
In this work, we demonstrated the possibility of knowledge fusion of several neural networks without an access to the original training data. The developed methods are applicable both for fully-connected and for convolutional networks. The methods were tested in classification tasks using publicly available datasets.
Noniterative methods (summation of weights and elastic weight consolidation) demonstrate the possibility of fast fusion of knowledge from several neural networks. The classification accuracy of the network with fused knowledge depends on the similarity of features of objects in the fused datasets: the classification accuracy increases when fusing knowledge from different areas. The classification accuracy of fANN obtained with WS and EWC methods is similar to one obtained from an ensemble of networks.
Generative iterative methods are slower than noniterative ones, however, they show higher accuracy in knowledge fusion. The accuracy of fANN obtained with generative methods is similar to the accuracy of the network trained on a joint set of training data. The quality of knowledge fusion is determined by the quality of reconstruction of original data by generative models and can be increased by improving VAE's generation quality [18] or by using different generative models [19]. In this paper, we presented generative methods only for shallow networks. In the future, we plan to extend the generative approaches for deep convolutional networks.
In this work, we considered examples of knowledge fusion between two ANNs. However, the proposed methods allow simple generalization for several networks and for networks with dissimilar architecture.
In addition to applications in machine learning, development of methods of interaction and information exchange between neural networks will be useful for other scientific areas such as neuro-