The metabolism of cells is the most basic and important part of human function. Neural networks in deep learning stem from neuronal activity. It is self-evident that the significance of metabolize neuronal network(MetaNet) in model construction. In this study, we explore neuronal metabolism for shallow network from proliferation and autophagy two aspects. First, we propose different neuron proliferate methods that constructive the selfgrowing network in metabolism cycle. Proliferate neurons alleviate resources wasting and insufficient model learning problem when network initializes more or less parameters. Then combined with autophagy mechanism in the process of model self construction to ablate under-expressed neurons. The MetaNet can automatically determine the number of neurons during training, further, save more resource consumption. We verify the performance of the proposed methods on datasets: MNIST, Fashion-MNIST and CIFAR-10.
The destructive methods have higher requirements for prior knowledge or pretrained model, take major resources in trained a network that is larger than necessary, it may get stuck in one of the intermediately sized optimal solutions @cite_7 . To construction, on the existing conditions to growth when add additional neurons or layers. In this paper, we introduce metabolism ideology into neural network, use proliferation and autophagy to gradually self-regulation in different model training stages.
@cite_7
2042481239
Abstract This paper introduces a new method called Dynamic Node Creation (DNC) which automatically grows BP networks until the target problem is solved. DNC sequentially adds nodes one at a time to the hidden layer(s) of the network until the desired approximation accuracy is achieved. Simulation results for parity, symmetry, binary addition, and the encoder problem are presented. The procedure was capable of finding known minimal topologies in many cases, and was always within three nodes of the minimum. Computational expense for finding the solutions was comparable to training normal BP networks with the same final topologies. Starting out with fewer nodes than needed to solve the problem actually seems to help find a solution. The method yielded a solution for every problem tried.