In this section, we compare our proposed DPP edge pruning with reweighting, DIVNET by [1]}, random edge pruning with reweighting, and importance edge/node pruning with reweighting on the MNIST [2]} and CIFAR10 [3]} datasets. We used the exact same network architectures as in Table 1 of [1]} for MNIST and CIFAR10, respectively. Following [1]}, we performed all pruning methods on the first layer.
We compare the number of parameters as \(k_e = \frac{k_n(d_{\text{input}} + h_2)}{h_1} - h_2\)  where \(k_e\)  is the number of edges kept for each node in edge pruning, and \(k_n\)  is the number of nodes kept in the hidden layer for node pruning; \(d_{\textrm {input}}\) , \(h_1\) , and \(h_2\)  represent the dimension of the input, size of the first hidden layer, and size of the second hidden layer, respectively. As in [1]}, \(h_1 = h_2\) . We trained our model until the training error reaches predefined thresholds (Table 1 in [1]}) and then perform the pruning. All edge pruning methods use our reweighting scheme in REF  while all node pruning methods use reweighting from [1]}. For hyperparameters and other details, see .
