extfieldsofstudy:
Physics
Computer Science

text:
Implementation of Basic and Universal Gates In a single Circuit Based On Quantum-dot Cellular Automata Using Multi-Layer Crossbar Wire

Quantum-dot Cellular Automata (QCA) is one of the most substitutes developing nanotechnologies for electronic circuits, as a result of lower force utilization, higher speed and smaller size in correlation with CMOS innovation. The essential devices, a Quantum-dot cell can be utilized to logic gates and wires. As it is the key building block on nanotechnology circuits. By applying simple gates, the hardware requirements for a QCA circuit can be decreased and circuits can be less complex as far as level, delay and cell check. This article exhibits an unobtrusive methodology for actualizing novel upgraded simple and universal gates, which can be connected to outline numerous variations of complex QCA circuits. Proposed gates are straightforward in structure and capable as far as implementing any digital circuits. The main aim is to build all basic and universal gates in a simple circuit with and without crossbar-wire. Simulation results and physical relations affirm its handiness in actualizing each advanced circuit.

INTRODUCTION
In last few decades, scaling the feature size and increasing the processing power have been successfully implemented by conventional lithography based on VLSI design [1].As the CMOS technologies approach its basic physical points of confinement, there has been broad examination as of late being developed of nanotechnology for future era IC. Gordon Moore has anticipated, in 1965 that the limit of a PC chip would become exponentially with time. From that point forward, the purported Moore's Law had represented the advancement and execution of microchips. Nanotechnology attracts much more attention from the researcher nowadays. As because the current CMOS technology faces different challenging problems like high power consumption and cannot ignored the difficulties in feature size reduction. A useful summarization report of future technologies has been published by ITRS [2]. Trusting on the unique properties of electronic devices at Nano level feature size, nanotechnology unlocks new prospects for computing systems and devices. QCA proposed by Lent [3], is an emerging technology that offers an innovative approach for computing at nano-scale by monitoring the position of a single electron. QCA technology has feature like high speed, extremely low power, fast switching speed [4] and much more dense circuit [5], which increases clock frequency of logic circuits and decreases the size and estimated power consumption of those circuits [6].XOR gate is presumably an imperative part of numerous advanced outlines of complex advanced circuits. The research article propose an advanced XOR gate by which any complex computerized circuit can be implemented, here we execute advanced parity checker circuit using crossbar multilayer. There are different clocking plans, for example, wave clocking which might be more reasonable for molecular QCA. Whatever remains of the research article is sorted out as takes after: In Section 2, the major of QCA innovation and outline methodologies is depicted including QCA clocking and QCA wire concept. Section 3 demonstrate the proposed work done over the earlier work. Segment 4 describes the design and simulation of combine gate using the multilayer gate as well as normal cell for QCA circuits. Section 5 concludes the paper.

QCA TECHNOLOGY AND DESIGN APPROACHES
Quantum technology has gradually applied in various fields, Quantum dot cellular automata are projected as a promising nanotechnology for future nano ICs. QCA technology is based on the interaction of bi-stable [7], [8] QCA cells constructed from four quantum dots. The cell is accused of the assistance of two free electrons which are able to tunnel between adjacent dots. These electrons have a tendency to possess antipodal destinations accordingly of their common electrostatic repulsion. Along these lines there exist two proportional enthusiastically negligible courses of action of the two electrons in the QCA cell, however no current directed out of the cell [9], [10] as appeared in Fig. 1.

QCA Logic Device
These arrangements are denoted as cell polarization P=+1 and P=-1, these two cell polarization P= +1 which represents"1" and P= -1 represents "0", binary information is encoded in the charge configuration of the QCA cell [11], [12].Not at all like conventional logic circuits in which data is exchanged by electrical current, QCA works by the Columbic communication that associates the condition of one cell to the condition of its neighbours, which results as data change.One of the essential logic gates in QCA is the majority voter (MV). The essential logic function in majority voter is shown in (1). MV can be acknowledged by just five QCA cells, as appeared in Figure 2(a) and 2(b) shown the logical block diagram of a majority gate. Logical AND ( Fig. 3 (a)) and OR ( Fig. 3 (b)) function can be implemented from the majority voter by fixing one input (control input) permanently to a 0 or 1. The inverter is another one basic gate in QCA and is shown in Fig. 3(c).    A few favourable circumstances can get utilizing QCA technology as is it "edge driven" means information is conveyed to an edge of a QCA block, that is assessed and yield at another edge. This moreover implies that no electrical cables need be steered inside. QCA frameworks ought to be low power, in light of the fact that there is no current flowing. Sufficiently just vitality needs to add to lift the electrons from their ground states. The required space for a circuit is little as in light of the fact that the extent of QCA cells are exceptionally little.

QCA Wire
A wire is implemented in QCA with a linear array of cells. A simple, 5-cell QCA wire is shown in Figure 4.

COPLANAR AND MULTI-LAYER CROSSOVER:
We have also investigated the tolerance of the so-called coplanar crossover, which is a signal crossing building block used in QCA circuits that operate on a four phase clock. The coplanar crossover can, under some circumstances, cross signals on one planar layer by taking advantage of certain symmetries between rotated and non-rotated cells. If these cells are perfectly realized, it can be shown that when placed adjacent to each other there will be no crosstalk between them, as shown in Fig. 6. Previous work has also examined the possibility of multi-layer QCA [Gin et al. 1999]. Using these multi-layer QCA circuits we can effectively cross signals over on another layer. To do this, we require a vertical interconnect. By stacking cells one on top of another we can transmit the signal to another layer where the signal is again transmitted horizontally. The multi-layer crossover is shown in Fig. 6(c).
There are two intermediate layers of cells to prevent any possible crosstalk between the two interconnects. For the multi-layer crossover simulations, the layer-to-layer separation was chosen to be 0.575 nm, 2.875 nm, 5.75 nm, and 11.5 nm for the 1 nm, 5 nm, 10 nm, and 20 nm cells respectively. The Coulombic interaction between stacked cells forces the cells in between to tend to the opposite polarization of the layers above and below. Unlike present CMOS integrated circuits, where metal layers are used to connect discontinuous sections of a circuit and cannot perform any logic functions, the extra layers of QCA are no different from the base layer. Cells lying in the extra layers are governed by the same Columbic interactions as those lying in the base layer. Thus, extra layers can accommodate computational circuit elements, not just wires. In this way, we believe that multi-layer QCA circuits can be made to consume significantly less area as compared to planar circuits. The layout for the coplanar crossover circuit is shown in Fig. 6(d), whereas the multi-layer crossover circuit is shown in Fig.6(e) Fig.6(d). The coplanar crossover circuit used in our simulations.
The simulation results for the coplanar crossover and the multi-layer crossover. Our simulation results indicate that each multi-layer circuit is significantly more fault tolerant than the equivalent coplanar crossover. The very poor performance of the coplanar crossover can be understood in terms of crosstalk between the vertical and horizontal wires. Given a perfect layout, the crosstalk between the two interconnects is zero. The single cell gap in the crossover wire drops the overall coupling between the two horizontal sections by a factor of 1/3232. The weak coupling renders the nearly floating output section of the horizontal interconnect highly sensitive to crosstalk. Even the smallest displacement can result in sufficient crosstalk between the two interconnects to cause the circuit to operate incorrectly. The robustness of the multi-layer crossover arises from the fact that there are no gaps anywhere along the length of the interconnects, and therefore none of the sections of interconnect are loosely coupled to each other. Although this building block presents a solution to crossing signals, it may be very difficult to realize in practice. Our objective in this work is not to evaluate the feasibility of such a building block. We are primarily interested in demonstrating that there are options available that will increase the overall displacement tolerance of the coplanar crossover [16].
A wire is implemented in QCA with a linear array of cells. A simple, 5-cell QCA wire is shown in Figure 7. These results indicate that the simple geometry of the wire has a beneficial effect on the fault tolerance. The primary fault with such a wire results from the addition of unwanted inversion as cells are displaced to the point where the kink energy that couples them goes from a positive to a negative value. Each cell that is coupled to its neighbor with a negative kink energy essentially acts as an inverter. On the other hand, two inverters in series cancel the fault.

QCA CLOCKING
Clocking is performed in one of two ways: zone clocking and continuous clocking. In zone clocking [15], each QCA cell is clocked using a four-phase clocking scheme as shown in Figure 9. The four phases correspond to switch, hold, and release and relax. In the switch phase, cells begin un-polarized and with low potential barriers but the barriers are raised during this phase. In the hold phase, the barriers are held high while in the release phase, the barriers are lowered. In the last phase, namely relax, the barriers remain lowered and keep the cells in an un-polarized state. An alternative to zone clocking, called continuous clocking, involves generation of a potential field by a system of submerged electrodes. The former clocking scheme is adopted in this paper since the CAD tool used for simulation supports zone clocking and further, prior works on adders are only based on this clocking scheme. Primitives in the QCA model consist of a wire, inverter and majority gate.

PROPOSED WORK
In this study we implement a QCA design for realisation of all gates using both simple cellular-dot automata and also using multi-layer crossover circuit. The novelty of this gate besides parameter like delay, number of cell count, majority gate and area is minimal in comparison to most other designs. Attempt is taken to design all the basic and universal gates with minimum cell count shown in figure 10(a) and 10(b). The comparison between various parameter of the circuit such as no. of cell count, time delay and total area consumption with the work available in literature has shown positive results. Complexity in terms of cell count, delay and area of QCA circuit can obtained by QCA Designer. The result indicate that no. of cell count is considerably low in our design shown in figure 10(c).

CONCLUSION
This article represents the design, implementation and simulation of basic and universal gates in a single circuit using quantum-dot cellular automata with multilayer crossover. This design is efficient in terms of cell count, area. Moreover, considering the less numbers of cell count, area and power consumption.


Neuron Pruning for Compressing Deep Networks using Maxout Architectures

This paper presents an efficient and robust approach for reducing the size of deep neural networks by pruning entire neurons. It exploits maxout units for combining neurons into more complex convex functions and it makes use of a local relevance measurement that ranks neurons according to their activation on the training set for pruning them. Additionally, a parameter reduction comparison between neuron and weight pruning is shown. It will be empirically shown that the proposed neuron pruning reduces the number of parameters dramatically. The evaluation is performed on two tasks, the MNIST handwritten digit recognition and the LFW face verification, using a LeNet-5 and a VGG16 network architecture. The network size is reduced by up to $74\%$ and $61\%$, respectively, without affecting the network's performance. The main advantage of neuron pruning is its direct influence on the size of the network architecture. Furthermore, it will be shown that neuron pruning can be combined with subsequent weight pruning, reducing the size of the LeNet-5 and VGG16 up to $92\%$ and $80\%$ respectively.

Introduction
Having today available a big number of large-scale datasets and powerful GPUs, deep neural networks have become the state-of-the-art in many computer vision, and speech recognition tasks [1,6,10]. They achieve high performance in many applications, e.g., scene and object recognition, object detection, scene parsing, face recognition, and medical imaging. However, they utilize high computational resources coming along with high memory cost [13]. For example, AlexNet and DeepFace have around 60M and 120M parameters, respectively [7]. Furthermore, they consume significant energy making their application on embedded devices difficult [7]. Containing a huge amount of parameters, deep neural networks may also be subject to over-parametrization. Thus, there could exist redundancies, and their generalization is not proper [12]. In general, networks with a small number of parameters generalize better extracting the important information of the data, rather than over-parametrized networks. Nevertheless, smaller networks are harder to train, since they are sensible to initialization [13].
Designing a network, i.e., setting the number of layers, neurons per layer, and parameters is typically still a "trial and error" process. Mostly, it depends on experience [1]. Moreover, training does not affect the structure of the network [8].

arXiv:1707.06838v1 [cs.CV] 21 Jul 2017
Several attempts have been developed for reducing the effect of the huge number of parameters, e.g., dropout [10], creating an optimal-sized network by adding additional regularizers, or pruning the network parameters [1,12,17]. The latter ones attempt to either remove edges or complete neurons from a network. However, most of these approaches require an expensive comparison of all neurons in the network or additional and expensive post-processing, e.g., the computation of the network's Hessian.
We propose an efficient and robust method for neuron pruning based on a local decision for reducing the number of parameters in a deep neural network. We will empirically show that pruning neurons rather than weights is essential for reducing the size of a neural network at runtime. The proposed approach for pruning neurons is based on the good performance of maxout units [5], which were developed for boosting the impact of dropout in training, and on their capacity to combine neurons for approximating more complex functions. It is assumed that redundancies typically exist in a neural network, so they also exist in a maxout unit, which combines the output of multiple neurons. Thus, pruning can be performed in a very local approach based on a single maxout unit.
The remainder of the paper is structured as follows: section 2 will discuss the related work in the field of parameter pruning for deep networks. In section 3 and 4, the maxout approach will be reviewed and an approach for pruning neurons will be introduced. Experiments on two datasets, the MNIST digits dataset and the labeled faces in the wild dataset, will be shown in section 5. Face recognition has been chosen as an application that is of special interest for embedded devices such as smartphones. The last section presents a short conclusion.

Related Work
Though deep neural networks are very powerful, they are known to be overparametrized, possessing millions of parameters [13]. This over-parametrization may cause performance deficits, e.g., poor generalization, overfitting, slow testing time, and enormous energy and memory consumption [3,7,8,12]. Therefore, reducing the size of the network by removing unimportant parameters, or designing optimal-sized networks becomes imperative. For those purposes, different attempts have been developed. These attempts can be grouped in constructive and destructive methods.
In constructive methods, neurons or layers are added to a trained shallow neural network. For example, in [17] a very deep convolutional neural network (CNN) is trained by continuously adding convolutional layers to an initial CNN of 11 layers for obtaining a better performance. However, the initial shallow networks must be properly trained, as the network can otherwise get stuck into a local optimum. Moreover, as the idea is to improve the network's performance by adding layers and neurons, the network's size increases, and redundancies could be introduced into the network.
In destructive methods, non-relevant neurons (neuron pruning) and/or parameters (weights pruning) of an initial deep neural network are removed, while maintaining its behaviour. The authors in [12] and [14] started the concept of pruning neural networks, both using a sort of relevance measure. In [12], parameters, with the smallest relevance in the network, which is computed by using the Hessian of the loss function, are deleted. In [14], complete neurons are deleted by using a relevance measurement based on the difference of the network's performance with and without the neuron. Nevertheless, especially with today's very deep neural networks with millions of parameters, computing the relevance of each neuron or parameter demands very high computational resources.
Different from the relevance measure methods, the authors in [7,8] prune weights by thresholding them. Afterwards, the network is re-trained for compensating the lost connections. One of the most prominent destructive methods is Deep Compression [7]. The authors reduced the storage required for a deep CNN by a factor of 35 and 49. They used a combination of three steps: weight pruning, weight quantization and Huffman coding. First, weight pruning is applied by thresholding the weights and thus setting them to zero. The remaining weights are then quantized, which reduces the number of bits for representing weights. Finally, a Huffman coding is applied. However, the networks are currently de-compressed for inference.
Recently, the authors in [1] determined the best number of parameters, by using a regularizer while training the network. The regularizer forces all the weights of single neurons to be zero. For testing, these dead neurons are removed from the network. Nevertheless, additional hyper-parameters must be determined for the regularization. The authors in [3] reduced the number of parameters to be learned by factorizing the weight matrices as a low rank product of two matrices: a static, and a dynamic matrix. First, they trained the static matrices as a general dictionary, obtaining a prior knowledge of the smoothness structures that are expected to be seen. Second, they fine-tuned the dynamic matrix, which are the weights to be learned.
Comparing the approaches, the destructive methods are more popular than the constructive ones as the networks are often easier to train. Good compression results are achieved by the deep compression approach in [7]. However, pruning weights has the disadvantage of rarely removing neurons from the network architecture. To make this clear, we show in Fig. 1 the relation between the proportion of remaining neurons in the network versus the proportion of pruned weights using the LeNet-5 [11] and the VGG16 [16] as examples. It clearly shows that neurons only get pruned at a very high compression ratio, which typically influences the networks performance. Thus, thresholding parameters allows for compressing a network but does not influence the size of the architecture. At runtime a sparse matrix library would be required, which efficiently evaluates the compressed network. Otherwise, the network must be de-compressed for inference. The zeroed weights are again stored in the memory, as in [7], which generates a waste in memory consumption as well as computational power.

Fundamentals
The proposed approach builds on a maxout architecture [5] for pruning the neural networks in a destructive manner. A maxout layer can be considered as a cross-channel pooling operation, performing a max operation between k adjacent neurons. These layers were designed to boost the model's averaging ability of dropout [10], thought for preventing overfitting, and to improve the optimization. Given an input layer X = [x 0 , x 1 , x 2 , ..., x N ] with N neurons, a maxout layer computes: where k is the number of neurons that are combined into a single maxout unit.
As the authors in [5] show, a maxout unit is a universal approximator. It combines k single neurons implementing a piecewiese linear function that can approximate arbitrary convex functions. So, in theory, the maximum of several neurons is able to approximate a more complex neuron. Moreover, the maxout unit becomes a sort of an activation function, replacing other activation functions, but with a factor of k smaller number of parameters. For example, two linear functions can implement a ReLU function, or five different linear functions can implement an approximation of a quadratic one, as shown in [5].

Compressing Networks with Maxout Architectures
The idea of the proposed approach is to use the maxout units and their model selection abilities for pruning entire neurons from an architecture without expensive processing. Thus, reducing the size and the memory consumption of a deep network. In some cases, the performance of the network may even increase as redundancies get reduced or eliminated. Following the assumption that redundancies exist in a deep neural network, it is assumed that if a network contains a maxout layer, redundancies will, also, exist in the maxout units. This is a valid assumption, since dropout and other regularization approaches cause the learn process to create different paths through the deep network, which yield similar outputs [5]. Using this premise, a reduction of the number of neurons in a maxout layer can be done without an expensive relevance measurement.

Neuron Pruning
For reducing the size of a CNN using maxout units, an iterative process is followed. First, a CNN with a maxout layer is trained. This maxout layer performs a max function among k adjacent neurons, reducing the amount of weights connecting with the next layer by a factor of k. So, placing this maxout layer after the one with the highest number of weights would be advisable. Second, by counting the number of times neurons become the maximal value in each maxout unit when computing a forward pass over the training dataset, the least active neurons of each maxout unit are removed from the network. Their effects are negligible with respect to other neurons. Third, the remaining neurons of the CNN are re-trained. After re-training, the process is repeated; in this case, the maxout layer performs a max function among k − 1 neurons, and so on. Fig. 2 shows an example for k = 4.
In comparison with [14], pruning neurons takes place locally, since relevance values are not computed depending on the network's output for each single neuron. The pruning in maxout architectures is therefore more feasible for very large networks with millions of parameters.

Weight Pruning
Having reduced the number of parameters in the network by pruning neurons from the maxout units of the network, further compression operations can be performed. Following the approach in [7,8], connections (weights) can be pruned in an additional processing step. Based on thresholding, edges with lower value than a threshold are set to zero. Thus, learning which connections are important and deleting the unimportant ones. By this weight pruning, the network becomes a sparse network. For pruning weights, a three-step procedure is followed, as proposed in [7,8]. Given the network that has been compressed by the proposed neuron pruning, the important connections are learned based on a global threshold. The threshold can be set such that as many connections as possible are removed without deteriorating the performance on a validation set. Second, weights below this threshold are deleted; that is, weights are set to zero. Third, the network is re-trained, learning the final weights.

Evaluation
An evaluation of both neuron and weight pruning is carried out for two different tasks: handwritten digit recognition, MNIST dataset [11], and face verification, LFW dataset [9]. The later is of special interest for embedded domains, e.g., in mobile phones. In general, the performance of the networks is evaluated with a varying percentage of pruned weights: after applying maxout, when pruning several neurons from the maxout units, and finally after applying additional weight pruning. While in the first task a very small LeNet-5 architecture is compressed, in the second task a large VGG16 architecture is compressed.
For the experiments, we chose k = 4 for the size of the maxout units as it allows for a fairly good compression and does not reduce the descriptiveness of the network compared to a network without maxout units. The neurons are then iteratively pruned from the maxout units and the network is re-trained after each pruning step.

Handwritten digit recognition
For the digit-recognition task, two networks, using the LeNet-5 architecture [11] with two convolutional layers, a fully connected layer and a softmax layer as a classificator, were trained. One network contains a maxout layer after the fully connected layer (LeNet-MFC), while the other has a maxout layer after the last convolutional layer (LeNet-MC). An iterative training following the steps in section 4.1 is executed using the MNIST dataset [11]. This dataset consists of 60000 handwritten-digit images (of size [28 × 28]) for training and 10000 images for testing. We used stochastic gradient descent (SGD) with a momentum of 0.9, weight decay of 5 × 10 −4 with inverse decay, a base learning rate of 0.01 that is iteratively reduced and a batch size of 64 for training. The networks were trained for 10000 iterations. Table 1 shows the classification accuracy for both networks with different fully connected layer sizes, with and without maxout (after the fully connected layer or the last convolutional layer). Pruning of one up to three neurons is evaluated. It shows also the percentage of pruned weights which do not remain in the network's architecture, denoted by p.w.%. In general, for both networks when using maxout and pruning neurons, the accuracy is maintained. The slight deviations of the accuracies of both networks with respect to the original networks are not significant based on a randomization test [15]. Moreover, the number of weights are considerably reduced with up to 70% for LeNet-MFC and 74% for  LeNet-MC. However, this reduction changes with respect to the position of the maxout layer. In LeNet-MFC, each neuron pruning step reduces the number of weights by 19.8%, because the neurons are pruned from the fully connected layer, which has the largest number of weights in the network. Besides, the maxout layer does not provide a considerable reduction, since it reduces the size of the softmax layer that has less number of weights compared with the other layers. In contrast, the weight reduction in LeNet-MC due to neuron pruning is just 1.4% per step, and it comes mostly from the maxout layer. In this case, the maxout layer reduces the fully connected layer instead, and the neurons are pruned from the last convolutional layer. However, in the last convolutional layer the number of weights is negligible. Following the neuron pruning, additional weight pruning, as discussed in section 4.2, can be applied. As mentioned in [8], neurons could also be pruned from the network if all their input weights are zero; that is, the neuron can be considered as dead. So, the number of neurons, and thus the number of weights, could be considerably reduced if a proper threshold is used. Nevertheless, analyzing the proportion of dead neurons versus the proportion of pruned weights, neurons do not become dead before pruning more than 98% of the weights, see Fig. 1(b). Consequently, weight pruning rarely prunes neurons. Thus, zeroed weights remain in the network as part of the neurons and the network's architecture does not change so that a sparse representation would be required at runtime [7]. However, assuming the usage of such a representation and for reducing storage size of the network, additional weight pruning is applied to both networks. As a basis, we use the networks after pruning three out of four neurons in the max- out units. The results in Tab. 2 show that the accuracy will not drop if less than 70% of the weights are thresholded. So, a total compression rate of 91% for LeNet-MFC and 92% for LeNet-MC, of pruned and zeroed weights, can be reached.

Face Verification
The neuron pruning was also carried out for a larger network for the purpose of face verification. The task is to verify whether two face-images portray the same person or not. For that purpose, the VGG16 network [16] was utilized, using The Visual Geometry Group Face Dataset (VGG face-dataset) as a training-dataset. This dataset is a large collection of face-images containing 2.6 million face-images from 2622 identities. It does not contain overlapping identities with standard benchmark datasets (LFW, YFT), so it is suitable for training. The VGG16 network, configuration D in [16], is a deep CNN with 16 layers: 13 convolutional layer, two fully connected layers, and a softmax layer. Analogous to the previous LeNet configurations, two configuration of VGG16 are used, in which a maxout network with k = 4 is added after the first fully connected layer (f c6), called VGG16-MFC, and after the last convolutional layer (conv 5), called VGG16-MC, see Fig. 3. The positions of the maxout layers are set after the layers with the most quantity of weights. Since, the connections between conv 5 and f c6 have 70.1% of the total amount of weights in the network and the connections between f c6 and f c7 have an additional 11.6% of the network's weights. The last three fully connected layers were fined-tuned for both networks. In the case of VGG16-MC, the conv 5 was also fine-tuned. We used SGD with a momentum of 0.9, weight decay of 5 × 10 −4 , three learning rates [10 −2 , 10 −3 , 10 −4 ], as [16], and a batch size of 128.
The network was tested following the procedure in [16], but using the restricted configuration of The Labeled Faces in the Wild (LFW) [9]. The LFW dataset is a standard benchmark dataset for face verification. It contains 13233 face-images from 5749 identities extracted from the Internet. Faces in images  were detected using the Viola-Jones face detector [9]. Faces are roughly centered, contain lesser noise but larger bounding-box than the ones in the VGG dataset. Besides, the Bray-Curtis distance (BC; [2]) was used instead of the Euclidean distance. Since, the BC distance works better for high-dimensional vectors in comparison with the Euclidean and the L1 distances [18]. The BC distance was measured between the descriptors of two face-images from a set of 3000 matched and 3000 non-matched pairs of images. Different from [16] and [10], the feature vectors from the crops of the image's corners were not utilized for computing the final descriptor, but only the crops from the image's centers.
If the BC distance is smaller than a threshold, then the two images portray the same identity. The Equal Error Rate (EER) [4,16] was used as the metric, which is defined as the value where the False Acceptance Rate (FAR), and the False Rejection Rate (FRR) are equal. Table 3 shows the EER for networks without a maxout layer and with a maxout layer with k = 4, as well as the results for pruning from one up to three neurons from each maxout unit. Similar to the previous results, neurons are pruned, and consequently weights are reduced, from the networks without affecting their performance negatively. In fact, the EER decreases by 1% and 0.47% for the VGG16-MFC and the VGG16-MC respectively. It is assumed that this improvement is produced by the elimination of redundancies in the maxout units. Based on a randomization test, the improvement in the VGG16-MFC is highly significant with p = 0.0026 [15].
The weight reduction changes depending on the position of the maxout layer and on the layer where neurons are pruned in the network. In VGG16-MFC, neurons are pruned from the largest layer in the network f c6 reducing the number of weights by 17.7% per each neuron pruning step. Moreover, the maxout layer reduces directly the size of the second largest layer f c7. In VGG16-MC on the contrary, neuron pruning does not affect considerably the size of the network, since it reduces a small layer conv5 compared to f c6. The weight reduction comes precisely from the maxout layer, which reduces the size of f c6. There is a difference of 7.45% between the weight reduction for both networks, since the size of the layer f c7 is never changed in VGG16-MC.
Additional to neuron pruning, weights from both networks were thresholded after pruning three out of four neurons in the maxout units with the results shown in Table 4. The network's performance will be, deeply, affected if more than 50% for the VGG16-MFC and 30% for the VGG16-MC of the weights are pruned. Nevertheless, a total compression rate of 80.1% for VGG16-MFC and 68% for VGG16-MC without performance deterioration can be reached.

Conclusion
We have presented an efficient approach for reducing the size of deep neural networks. This approach prunes entire neurons and thus reduces the number of weights in neural networks. It uses maxout units for combining k single neurons into complex ones. A maxout layer reduces the number of weights between two adjacent layers by k. By using these maxout units, the network's performance is not negatively affected, since they boost the dropout benefits reducing redundancies in the network. Within these maxout units, neurons are pruned based on a local and non-expensive relevance measure. This relevance measure depends on the number of times neurons are maximal for each of the k adjacent input-neurons per maxout unit. It differs from previous relevance measures, because it does not depend on the overall network's performance with and without individual neurons [14]. In general, this approach does not require expensive post-processing, only a single re-training after pruning. The performance of this reduction approach depends strongly on the position of the maxout layer in the network. As inputs from maxout units are the neurons to be pruned, it is advisable to place the maxout units after the largest layer in the network, because neurons in this layer have large numbers of weights compared with neurons in other layers. So, pruning these neurons out of the network is favorable. By comparing the number of pruned neurons and network's performances between the aforementioned approach and weight pruning, the last one does not delete entire neurons, but rather sets weights to zero. Therefore, the architecture's size is only implicitly reduced, and the memory footprint remains equal without a sparse representation. The proposed approach allows to reduce a network's size by 61 − 74% on an architectural level and without affecting the network's performance. Assuming a sparse representation, a combination of the proposed neuron pruning with additional weight pruning allows for reducing the size of a network by up to 92%.