Deadwooding: Robust Global Pruning for Deep Neural Networks

The ability of Deep Neural Networks to approximate highly complex functions is key to their success. This benefit, however, comes at the expense of a large model size, which challenges its deployment in resource-constrained environments. Pruning is an effective technique used to limit this issue, but often comes at the cost of reduced accuracy and adversarial robustness. This paper addresses these shortcomings and introduces Deadwooding, a novel global pruning technique that exploits a Lagrangian Dual method to encourage model sparsity while retaining accuracy and ensuring robustness. The resulting model is shown to significantly outperform the state-of-the-art studies in measures of robustness and accuracy.


Introduction
As deep learning models evolve and become more powerful, they also become larger and expensive to store and execute. This aspect challenges their deployment in resource constraint settings, including embedded systems and Internet of the Things, which require lightweight and real-time executable models. To address this challenge, recent studies have developed a variety of methods to prune the relatively less important parameters of a neural network followed by a fine-tuning step that retrains the retained parameters to ensure high accuracy [45,1,54]. The importance of training large, often over-parametrized models, followed by a pruning stage, has been emphasized in a wealth of studies [55, 22,27] showing that the resulting pruned models perform overwhelmingly better, empirically, than dense models (of size comparable to the pruned models) trained from scratch.
While pruning can allow retaining high accuracy at a reduced model size, it has also been shown that the pruned models are often severely more susceptible to adversarial examples, when compared with the original models [20] (see Appendix B for a detailed review). Since these size-reduced models are often deployed in security-critical contexts, including autonomous driving and home surveillance systems, poor model robustness can have disastrous consequences [4,41]. There is thus a critical need to corrobrate that pruned models ensure both accuracy and robustness to adversarial attacks.
This paper addresses this need by introducing Deadwooding, a novel robust global pruning paradigm which leverages Lagrangian duality to achieve the combined objective of model sparsity and robustness. The paper contributions are summarized as follows: (1) It casts the problem of robust model pruning to a constrained optimization problem that minimizes the empirical risk under a desired pruning degree and tolerance to adversarial attack. The resulting non-linear learning task is solved using Lagrangian dual learning method to compute an approximate solution that encourages the satisfaction of the desired pruning and robustness constraints. (2) While pruning, robustness is imparted using a novel proxy estimate that aims at increasing the distance between the projection of input samples and the decision boundary, thus, reducing their vulnerability towards adversarial attack. Notably, this method is independent of any adversarial attack model. (3) While global pruning retains accurate pruned models it may also generate disconnected networks at high pruning amounts [7]. To address this limitation, Deadwooding uses a dedicated loss term to ensure network connectivity while globally selecting the parameters to be removed. (4) Next, the paper proposes a novel adaptation of knowledge distillation that incorporates adversarial training for the fine-tuning step. The resulting process retrains the non-pruned parameters to improve both accuracy and adversarial robustness. (5) To aid adversarial training during the fine-tuning step, the paper also introduces FGSM-looping, a novel variation of the fast gradient sign method (FGSM) [40], which achieves comparable performance to the state-of-the-art (SoA) attack generation approaches with a significantly reduced computation time. (6) Finally, differently from the SoA robust pruning approaches that perform robust pruning only on already robust models, Deadwooding produces robust pruned models irrespective of whether the original complex model has undergone adversarial training or not.
The extensive experimental evaluation on several settings and a benchmark dataset: CIFAR-10 illustrates significant improvements over state-of-the-art models. A detailed background description of this paper's relevant terms and concepts are provided in Appendix B.

Related work and limitations
The importance of model parameters towards model inference can be evaluated using several factors, including but not limited to the magnitude of their weights. The parameters with less magnitudes have been shown to have the least impact on the model's output [50,5] and prediction accuracy. The majority of the SoA pruning studies [22,45,54,21,26,44,42] (including the ones referred to in this paper) use magnitude-based parameter pruning. While neural network model pruning has received much attention, the literature addressing the combined objective of pruning and robustness to adversarial examples is exceedingly more sparse. Notably, Sehwag et al. [45](LWM) removed the model weights with the lowest magnitude iteratively while fine-tuning the model for adversarial robustness in each iteration, pruning a maximum of 90% parameters. Zhang et al. [54] approached pruning using the alternating direction method of multipliers (ADMM), achieving a maximum of 71% pruning. Recently, Sehwag et al. [47] presented Hydra, which used an adversarial training objective during the pruning training step and fine-tuned the resulting pruned model to ensure high accuracy, achieving a maximum of 99% pruning.  Although these works present important contributions to the state of the art, they all have the following two limitations: (1) To avoid creating disconnected sub-networks, these methods all perform layer-wise pruning, which restricts the pruning amount for each layer to a fixed percentage. As we will show next, this strategy induces suboptimalities to the model performance.
(2) Existing robust pruning approaches assume that the original, complex model to be pruned, is already an adversarially robust model, and attempt at preserving such property.
Limitation (1) is expressed in Figure 1, which reports the amount of parameters removed, in each layer of VGG-16 at 99% pruning for three approaches: (i) a generic least-weight magnitude based global pruning method (described in more details in Appendix B), (ii) Hydra by Sehwag et al. [47], which performs layer-wise pruning, and (iii) Deadwooding. Notably, in both global pruning approaches (Deadwooding and generic), the initial feature extraction layers and the final decision-making layers undergo less pruning than the middle layers. The generic global pruning removes parameters based on their magnitudes and the importance of a parameter is considered to be directly proportional to its magnitude [22]. Thus, it indicates initial and final layer's parameters are more critical for accurate classifications. But, generic global pruning removes the layers 10-12 completely, resulting in a disconnected network. While Hydra [47] generates a connected pruned network by removing an equal amount of parameters from each layer, it also removes more parameters in the critical layers than Deadwooding; thus, Deadwooding generates pruned models having higher accuracy and robustness.

Problem settings and model pruning goals
The paper considers datasets D of n individual data points (X i , Y i ), with i ∈ [n] drawn from an unknown distribution. Therein, X i ∈ X is a feature vector and Y i ∈ Y is a label. Given a classifier M θ : X → Y, where θ ∈ R k is a real valued vector describing the model parameters, the goal is to learn a reduced classifier M θ ↓ : X → Y with θ ↓ ∈ R k and k k, which is substantially smaller than the original classifier while retaining high benign accuracy and ensuring robustness. The model quality is measured in terms of a non-negative loss function : Y × Y → R + , which, in this work is the cross-entropy loss [31]. The problem is that of minimizing the empirical risk function: The resulting model performance over the test set is referred to as benign accuracy.
Adversarial robustness refers to a model's ability to resist to adversarial examples [49]: inputs that are produced by perturbing them slightly with a perturbation budget ( > 0) in a way that causes them to be misclassified by the model. The prediction accuracy of the model against adversarial examples [11,2,30] refers to its adversarial robustness, and it is measured in terms of an adversarial loss function, as defined by Sehwag et al. [47]: Therein, ⊕ is an adversarial attack generation method and adv is an adversarial loss assessing the effect of the adversarial attack.
Problem statement. Given a pruning target k , the desired goal is to learn a reduced model M θ ↓ with θ ↓ ⊂ θ that has high benign accuracy while being robust. This ideal objective is formalized as: where b is a k-dimensional binary vector. In the above, the constraint (3b) imposes the pruned model to be robust to adversarial attacks and (3c) defines a capacity constraint used to construct the pruned model. The optimal pruned parameters θ ↓ can be retrieved as θb.

Robust pruning: a Lagrangian dual view
Notice that the ideal problem above is an NP-hard Mixed Integer Non-Linear program. Finding a reduced model M θ ↓ through Program (3) is thus not computationally viable, therefore, this section specifies a Lagrangian relaxation of the program above and proposes a Lagrangian Dual approach to enforce the problem constraints during training.
In Lagrangian relaxation, the constraints of an optimization problem are relaxed into the objective function using Lagrangian multipliers to capture the penalty induced by violating them [16]. The Lagrangian function of Problem (3) is: where parameters θ and D are omitted for clarity of explanation. Therein, variables b have been relaxed in [0, 1] ⊆ R and the terms λ a and λ p describe the Lagrangian multipliers associated, respectively, with the adversarial loss and the pruned model capacity constraints.
For multipliers λ = (λ a , λ p ) solving the optimization problem above produces an approximation M θ ↓ (λ) of M θ ↓ . The Lagrangian dual computes the optimal multipliers, i.e.,: to obtainM * = M θ ↓ (λ * ) , the strongest Lagrangian relaxation of Problem (3). LearningM * relies on an iterative scheme that interleaves the learning of a number of Lagrangian relaxations (for various multipliers) with a subgradient method to learn the best multipliers: where ρ > 0 is the Lagrangian step size. In the implementation, step (6a) is approximated using a gradient-based method.
While a Lagrangian relaxation of Problem (3) does not obviously ensure finding the minimizer which satisfies the problem constraints exactly, as we will show in Section 6, the Lagrangian dual method exploited in this paper is highly effective in building robust pruned models. The Lagrangian dual learning method to prune a model is a novel contribution of this work.

Deadwooding
Deadwooding consists of two phases: (1) Model Pruning and (2) Fine-tuning. First, the Lagrangian Dual based pruning approach forces the least useful k−k k % (i.e., desired amount of pruning) network parameters to approach zero while ensuring high accuracy and robustness. These parameters are then removed with minimum impact on the model accuracy (see Section 5.1). Next, an adaptation of the knowledge distillation is used as a fine-tuning step (see Section 5.2).
The pruning step implements the robust pruning objective through the Lagrangian dual method described in Equation (6). The process is described in Algorithm 1. Given the input dataset D, the optimizer step size α > 0, and a Lagrangian step size ρ t , the Lagrangian multipliers are initialized in line 1. The training is performed until the required number of parameters are below a predecided threshold W (line 2), and each epoch t optimizes the reduced model parameters θ ↓ = bθ of the classifier M bθ(λ t ) using the Lagrangian multipliers λ t associated with current epoch (lines [3][4][5]. Finally, after each epoch, the Lagrangian multipliers are updated according to a dual ascent rule [8] (lines 6 and 7). The resulting model statisfies the required capacity constraints (given the imposed threshold factor). This training step is thus followed by the removal of the remaining k − k parameters with corresponding b values smaller than W .
The following subsections describe the three components of the pruning loss Equation (4) (Algorithm 1 line 5) and their relation with achieving robustness and pruning.
Adversarial robustness proxy. To incorporate robustness during the pruning step, Deadwooding uses an adversarial proxy loss function that aims at maximizing the distance of the projection of the input samples to their decision boundary. Intuitively, samples farther away from the decision boundary are less sensitive to input perturbations under the lens of the model output. Following Tran et al.
[51], we approximate the distance of an input sample X to the decision boundary for a K-class classifier using the output logit values: The term M θ,k (X) ∈ [0, 1] denotes the k th logit value associated with input X given the model with parameters θ. The above rely on the fact that, samples close to the decision boundary are associated with uncertain model decisions. Thus, the logit values of samples that are close to the decision boundary tend to be similar and approach 1/K, as the samples get closer to the decision boundary. This leads to a high adv value. Conversely, samples which are farther away from the decision boundary, are predicted with higher confidence and, thus, one of the logit values will be much higher than the others, leading to a low adv value. Thus, minimizing L adv leads to learning models which separates the input samples from their decision boundary as much as possible.
Pruning constraint proxy. Note that the pruning constraint b 2 − k introduced in the Lagrangian loss function effectively models a Ridge regularization. The regularization term enforces sparsity in the pruned network. The Lagrangian dual model enforces this regularization term with strength λ p which is proportional to the constraint violation (difference to the desired pruning amount).
Global pruning. As mentioned earlier, existing robust pruning studies are limited to layer-wise pruning, since global pruning may generate disjoint subnetworks in high pruning regimes (see Figure  1). A simple, yet effective way to address this limitation is by adopting the loss term L (which measures the prediction accuracy) in Equaion (4). In the absence of an active path, the pruned model's loss L λ will increase significantly. During the model pruning step, the pruned network maintains an active path through the retained parameters θ ↓ = bθ to minimize L λ , thus enabling global pruning while also ensuring a connected pruned network.

Fine-tuning step
Conventionally, pruning is followed by a retraining step over the non-pruned parameters (known as fine-tuning) that helps to relearn the information lost by removal of the pruned parameters. Interestingly, Chen et al. [12] showed that a pruned model fine-tuned using knowledge distillation (KD) exhibits better prediction accuracy than one subject to natural re-training. Vanilla KD [24] aims to transfer the knowledge learned by one (teacher) model, to another (student) model. The student is trained to achieve high accuracy mimicking the teacher model by learning response-based knowledge.
The KD loss function L KD has two components: a hard loss, L, and a soft loss, L soft . The former accounts for student model's prediction errors whereas the latter enables the student to learn the relation among different classes through a softmax computed at high temperature values (see Appendix B for additional details). In its fine-tuning step, Deadwooding uses a modified KD step, which includes an additional component to account for adversarial loss L adv ( , θ; D) defined as in Equation (2). The original, complex, model is referred to as the teacher and the pruned model as the student. The overall fine-tuning loss function is defined as: where θ and D have been removed for clarity of explanation and α, β, γ are trainable hyper-parameters.
In the experiments, these values were obtained using the hyper-parameter tuning tool Optuna [3]. It is important to note here that Lagrangian-dual approximation is applicable to only constrained optimization problems and thus, not directly applied to optimize the coefficients of Equation (8).
The fine tuning step requires the generation of adversarial examples to evaluate the loss component L adv . To do so, Deadwooding uses a simple adaptation of FGSM, named FGSM-looping, that instead of using a single perturbation value to generate the adversarial examples, it iterates through a set of values E = ( 1 , 2 , . . .) with each i ∈ (0, max ] with max being the maximum perturbation allowed. This helps to generate multiple perturbed samples around a point within an L ∞ -ball of radius max , whereas vanilla FGSM generates adversarial samples at the boundary of the region of interest. The evaluation illustrates that using FGSM-looping during fine-tuning reduces computation time while achieving eba and era comparable to SoA approaches. Further details are provided in Appendix D.3.

Experiments
This section compares the Deadwooding's performance, in terms of accuracy and adversarial robustness, against the current SoA pruning method Hydra [47]. Notice that Hydra is the only SoA robust pruning baseline available and, thus, the most relevant to our analysis.
Settings and evaluation metrics. The evaluations uses the benchmark dataset: CIFAR-10 and VGG-16 network architecture. Further detail on experimental setup, system, and configurations are discussed in Appendix A.
This paper uses two metrics to measure the performance of pruned networks: Empirical Benign Accuracy (eba) and Empirical robustness accuracy (era), which are defined by the SoA baseline  Table 1: Comparison of pruned models obtained using Hydra and Deadwooding starting from robust original model (M r θ ) VGG-16 trained for CIFAR-10 based on eba and era. ∆ represents the performance gain by Deadwooding. [47]. eba is the percentage of correctly classified benign (i.e., not adversarially perturbed) examples, where era is the percentage of correctly classified adversarial examples generated with an adversarial attack ⊕. Following the observation by Carlini et al. [10], suggesting that using the same ⊕ to attain and evaluate robustness of a network can lead to biased results, Deadwooding uses FGSM-looping as ⊕ in Equation (2) to impart robustness during fine-tuning step, and PGD attack [40] as ⊕ to evaluate the pruned model's robustness (i.e., era). However, the paper also evaluates the pruned models' robustness with several different (and stronger) attacks such as AutoAttack [13], FAB [14] and DeepFool [35] as ⊕. These extended results are reported in Appendix E.2.

Robust network pruning evaluation
This section evaluates the ability of Deadwooding to retain accuracy and ensure robustness. Table  1 compares Deadwooding with Hydra when pruning robust original models. Table 1 reports mean values for 10 different repetitions and for three pruning constraints: 90, 95, and 99, indicating the percentage of parameters to be pruned from the original networks (PT(=0%)).
Therein, M r θ ↓ , and M r(H) θ ↓ denote the pruned models resulting from Deadwooding and Hydra starting M r θ , respectively. Key observations from Table 1 are: • Deadwooding extracts pruned models that are more robust than their original complex counterparts. Deadwooding achieves a mean improvement in by era 5.63 (∆(M r θ ,M r θ↓ )) percentage points with respect to the original models (PT=0%).
• Deadwooding produces highly accurate pruned models. While it is folklore that pruning, in general, reduces overfitting and, thus, may result in accuracy improvement [6], extreme pruning may generate networks with insufficient parameters to model the classification function, resulting in a loss of eba. Surprisingly, the reported results show that Deadwooding achieves pruned models having eba comparable to original model. This is significant, as large decreases in eba may limit the usability of the pruned models, especially in applications such as health monitoring and autonomous systems.
Deadwooding's comparison with SoA. A detailed comparison is discussed below: Comparison with Hydra: As shown in Table 1, while pruning a robust complex model, Deadwooding achieves a significantly higher era and eba as compared to Hydra. Notably, at a 99% pruning ratio, Deadwooding achieves the highest gain. As discussed in Figure 1, Hydra's layer-wise pruning removes 99% parameters even from the important layers; hence some layers may not have enough retained parameters to model the classification tasks effectively. In contrast, Deadwooding's global pruning retains larger percentages of parameters in important layers while pruning more (up to 99.93%) in less significant layers, hence achieving consistently high robustness and accuracy.
Approach comparison with most recent SoA baseline Hydra. Finally, the section shed lights on the contrast in performance attained by Deadwooding when compared to Hydra by focusing on analyzing the key conceptual differences between these two approaches.
Firstly, Deadwooding prunes the original complex network's parameters globally while maximizing the projection of input to the decision boundary distances to enforce adversarial robustness. This means that it considers the importance of the parameters from the complete network's perspective and its attainment of robustness during pruning is not dependent on any specific attack ⊕. In contrast, Hydra performs layer-wise pruning while maximizing adversarial robustness against the PGD attack.
This means it needs to remove a relatively higher number of parameters from the important layers, and its robustness is dependent on the specific attack adopted (i.e., PGD). Our analysis shows that the parameters pruned by Deadwooding are 10 3 times smaller, in magnitude, than the retained ones. In contrast those removed by Hydra are just 10 times smaller in magnitude than the retained ones.
Finally, Hydra uses natural re-training as the fine-tuning step, and Deadwooding uses a modified Knowledge Distillation (with additional enforcement of adversarial robustness), resulting in higher era, while achieving similar (marginally higher or lower) eba. These characteristics allows Deadwooding to extract robust and accurate pruned models, even at a high pruning ratio (99%).

Delving Deeper into the Deadwooding
This section evaluates the importance of Deadwooding's different components and their characteristics. The evaluations are performed on VGG16 trained over the CIFAR-10 dataset.
Deadwooding's pruning & fine-tuning components. We make the following observations:  • Inclusion of benign accuracy through loss L. As discussed in Section 5.1, the inclusion of benign accuracy in Equation (4) enables global pruning by ensuring a connected network with valid input to output layer path. Table 2 clearly shows its importance by evaluating eba and era on a modified loss which ignores the term L from Equation (4) in the pruning step. Note that the resulting pruned models become disconnected, which thus causes the low scores. Additional analysis are shown in Appendix D.1.
• Modified Knowledge distillation fine-tuning. Chen et al. [12] suggests that applying vanilla KD as fine-tuning step restores the benign accuracy in the pruned model. Additionally, Papernot et al. [36] showed that adversarial KD (L soft &L adv ) makes the neural network models further robust. Table 3 shows that the modified KD fine-tuning (Section 5.2) achieves pruned models of high eba and era, whereas vanilla and adversarial KD as fine-tuning achieve high eba but lower era. Thus, the proposed modified KD step effectively achieves highly robust and accurate pruned models.   • Using Distance between Projection of Samples and Decision Boundary to impart robustness. This section discusses the effectiveness of using distance to decision boundary as a proxy measure to robustness (Equation (7)). Figure 2 reports the average distance d, d(X) = 1 − adv (X) (computed as described in Equation (7)) for the test data.
Higher d values correspond to a higher distance of the samples to the decision boundary. The figure reports the distance to boundary d of the benign and adversarial samples at different pruning amounts for the robust VGG16 model with the CIFAR10 dataset. Note that, on average, correctly classified benign and adversarial samples have higher d than the incorrect ones.
In low pruning ratios (20-50%), the model goes through the pruning step with distance constraint (Equation (7)) while retaining a large portion of the parameters. In this context, the pruned model is still quite large and the d of the benign and adversarial samples are high.
However, the model complexity reduces significantly at very high pruning ratios (90-99%), typically associated with simplified decision boundaries. Hence, the d values are lower than those obtained in lower pruning ratios. Notably, eba and era are relatively high in high-pruning ratios (90-99%). This is due to the fine-tuning step (Section 5.2) that re-adjusts the decision boundaries utilizing the retained parameters. Figure 2 shows the robustness attained through increasing d in different pruning ratios.
The results clearly demonstrate the need of fine-tuning, specifically in high pruning ratios.
• Lagrangian dual model and Parameter Weights. According to the magnitude-based pruning assumption [50,5], in a high pruning ratio, the retained parameters should have significantly higher absolute weights, containing substantially more impactful information, compared to the removed parameters. Notably, in our experiments (not tabulated due to space constraints), after the model pruning step (Section5.1) for 90%, 95% and 99% pruning ratios, the mean absolute weights of the retained parameters are 1100, 1721, 2381 times higher than that of each removed parameter. Additionally, removing the least absolute weight parameters reduces the models' accuracy by at most 0.1%, thus, demonstrating their minimal impact. Finally, the Lagrangian dual model adopted achieves highly accurate and robust pruned models, which shows that the retained high mean absolute weight parameters are highly impactful. Thus, the Lagrangian dual model enables effective magnitude-based pruning [50,5], that is in line with our pruning objective in Equations (3a)-(3d).
Miscellaneous additional evaluations for Deadwooding are discussed in Appendices E and F.

Concluding remarks and limitations
This paper was motivated by the recent observations about the brittleness of pruned models to adversarial attacks. To contrast these observations, the paper presents a novel pruning approach named Deadwooding that achieves highly sparse and robust models while maintaining original complex models' accuracy. Deadwooding exploits a novel Lagrangian Dual learning method to encourage model sparsity while enforcing robustness. The resulting models were shown to outperform state-of-the-art pruning approaches significantly. Notably, Deadwooding operates effectively under extremely high pruning-ratios (up to 99%), and, in contrast with some previous work, it does not leverage any biases or characteristics in the data or domain to attain sparsity and robustness.
We note that the proposed method is intended to compute pruned models which are robust within a certain bounded region guided by the perturbation budget max , as suggested by [10]. However, it does not claim high robustness outside the L ∞ region of radius max .While this is a common shortcoming of arguably all practical robust ML models, we believe the is an important avenue of future research.
Despite these avenue of improvement, the authors strongly believe that the presented pruning method will pave the way to the use of robustly pruned ML models in practical safety-critical applications in resource-constrained environments. [

A Expermental Details
The following section describes the details of the experimental setup: Datasets To demonstrate the effectiveness of our approach Deadwooding, the evaluations are carried across three publicly available benchmark dataset: CIFAR-10 with maximum perturbation amount ( max ) of 8/255, respectively. For fairness of comparison, the dataset and the corresponding max are kept consistent with state-of-the-art [47,54]. These datasets can be downloaded and accessed through open-source Torchvision library [32].
CIFAR-10 is a dataset of 60000 images evenly distributed among 10 mutually exclusive classes, out of which 50000 form the training set and 10000 form the test set. The train and test set reflect the same fair distribution among the 10 classes.
Metrics: Benign Accuracy and Robustness. Empirical Benign Accuracy eba and Empirical Robust Accuracy era are the two metrics used to measure the performance of a model quantitatively. Benign accuracy eba refers to the measure of the accuracy of prediction for the benign (unperturbed) test samples. Several definitions of adversarial robustness exist in literature [15]; however, to maintain consistency, we constrain our evaluations to the definition provided by the SOA baseline [47]. Therefore, Empirical Robust Accuracy era refers to the measure of the accuracy of models' prediction for the perturbed samples, which are generated using the test set. In this study, for generating perturbed samples, we used FGSM-looping during training and the PGD [40] approach while evaluating (i.e., test) the models to compare with the baseline state-of-the-art approaches. Additionally, we also evaluate the models for stronger attacks like Auto-Attack [13], FAB [14] and Deepfool [35] and the results are presented in Appendix E.1.
System Configuration All evaluations are carried over a system with x86_64 architecture AMD Ryzen 16-core processor, 115 GiB RAM hosting a 16GiB GPU.
Model Training and parameter configuration Deadwooding makes use of model training at two stages: pruning and fine-tuning.
During the pruning stage, we leverage Lagrandian dual based optimization (discussed in Section 4) to compute an approximate solution to achieve a combined objective of maximizing robustness and sparsity while maintaining SoA accuracy. The λ coefficients for the different components of the loss function are learned within the training process. With each epoch, the model gets more sparse and the amount of sparsity serves as the stopping criterion for this training phase.
For fine-tuning, we use modified knowledge distillation which has an additional term for adversarial loss in its loss function (discussed in section 5.2). We used Optuna [3], a parameter-tuning tool, to obtain the best values for the coefficients of three loss components: α, β andγ with the objective to maximize era. Optuna uses a Bayesian Optimization algorithm called Tree-Structured Parzen estimator to arrive at the optimum set of values. For instance, pruning 99% parameters of VGG-16 trained for CIFAR-10 dataset used α = 0.351, β = 0.526 and γ = 0.240. The resulting pruned model has an eba = 80.87 and era = 53.1. We ran the fine-tuning step for a maximum of 100 epochs with early-stopping patience of 30 to avoid over-fitting.
For fairness of Comparison with SoA The results for Sehwag et al. [47] (Hydra) are generated using codes available at public git repository Sehwag et al. [46], respectively. The original models used for evaluations are taken from the checkpoints available at the public git repository Sehwag et al. [46] provided by Sehwag et al. [47].

B Background Discussion
This section discusses the details of various terms used in this work. Section B.1 describes model pruning and the most prevalent ways to achieve it [21,54,45,47]. Section B.2 described adversarial attacks and how adversarial training is used to achieve robustness against such attacks. Deadwooding uses a modified form of knowledge distillation during the fine-tuning stage; thus, Section B.3 is used to discuss the details of vanilla knowledge distillation.

B.1 Model Pruning
Initial efforts to optimize the inferences made by the machine learning models led to a considerable increase in their size in terms of the number of parameters, which require enormous amount of disk space. For instance, the number of parameters required for VGG-16 trained on CIFAR-10 are 138M. Parameter pruning techniques tend to achieve the effect of a compressed model by making the model parameters sparse. The conventional method of parameter pruning is to remove the connections (i.e., parameters or neurons) which contribute the least towards the model's inference [45,54]. In literature, the importance of the model parameters is measured in terms of the magnitude of their value [50,5]. However, at high pruning amounts, such a removal leads to a loss in accuracy. The retained model parameters are then fine-tuned (i.e., re-trained) to re-gain the lost performance [22].
Global vs Layer-wise Pruning: In literature [21,54,45,47], pruning has been done either layerwise or globally. Layer-wise pruning checks for the least contributing connections belonging to a single layer at a time, whereas global pruning considers all the model connections simultaneously for removal of least significant parameters. It has been shown recently that for the same amount of pruning, the model pruned using global pruning exhibit better accuracy of prediction than a model pruned using layer-wise pruning [1]. Since global pruning does not restrict the fraction of parameters pruned or retained for each layer, the network pruning solely based on the magnitude of weights may result in disconnected networks for some instances. These disconnected networks cannot regain their performance, even after fine-tuning. To avoid such circumstances, state-of-the-art approaches Sehwag et al. [47,45], Zhang et al.
[54] followed layer-wise pruning, which removes a fixed amount of parameters from each layer so that each layer retains some of its parameters. We, however, prune the networks globally, resulting in different pruning amounts at each layer. Moreover, while globally pruning the network, we ensure to retain the model's accuracy, thus, resulting in at least one valid path from the input layer to the output.

B.2 Adversarial attacks and robustness
Goodfellow et al. [18] demonstrated that small perturbation to input samples can result in incorrect model inference with high confidence. A perturbed sample is generated by adding a well-crafted noise to the sample so that an oracle (or human observer) perceives it as the original. The perturbed samples which get misclassified by the target model are considered adversarial samples [18]. Several adversarial attacks like FGSM [18], PGD [30], DeepFool [34], etc. have been proposed in the literature to compute worst-case adversarial samples with minimum perturbations. A model which can correctly classify these perturbed samples is said to possess adversarial robustness.  Pruning reduces robustness Figure 3 shows a depiction of the projection of samples belonging to two classes, each surrounded by L ∞ region of a fixed radius (known as perturbation budget), where a corresponding perturbed sample can lie. Figure 3 (a) represents a complex decision boundary learned by the original complex model. Lesser number of parameters limits the ability of a pruned model to learn this complex function and the decision boundaries thus learned are quite simple. Figure 3 (b) shows that for a pruned model, when a perturbation is introduced, the samples closer to the decision boundary are more prone to incorrect classification because of the simplified decision boundary.

B.3 Knowledge Distillation (KD)
Vanilla KD aims to transfer the knowledge learned by one model, called as teacher model, to another model, named the student model. The student model is trained to achieve a combined objective of high accuracy and learning the teacher model's inference space distribution. Minimizing the loss functions L KD is the key behind the training of the student model.
Here θ t and θ s represent the parameters of the teacher model and the student model, respectively. x is an input sample, y is the designated label, andŷ is the prediction made by the student model. Thus, L aims to achieve accuracy by minimizing the difference between the true labels and the predictions made by the model. Softmax of model output is computed using the formula f T (θ, x) = e M θ (x)/T Σ n k=1 e M θ (x)/T , by substituting T = 1. Using higher values of T, allows for smoother probability distribution in output which reveals the relation among all the classes. In Equation 9, f T (θ t , x) and f T (θ s , x) correspond to the logit values generated using softmax at temperature T by the teacher and student models [25]. L soft enables the student model to learn response based knowledge of the teacher model, provides better generalization, and helps to optimize the student model leveraging a small amount of training data [19]. Several pruning approaches leveraged KD to fine-tune the retained pruned network [1,38]. Notably, a previous work [36], used knowledge distillation to enhance the robustness of a model having the same architecture as that of the teacher model.

C Related Works: Robust Pruning
Initial efforts that used model pruning as a model compression mechanism targeted maintaining accuracy while pruning the redundant parameters. Han et al. [21] used the least-weight-magnitudebased global pruning mechanism and achieved a maximum pruning amount of 34%. Li et al. [26] focused on pruning conv-nets by removing filters instead of independent parameters for convolutional layers pruning a maximum of 38.1% parameters. Zhang et al.
[54] defined the pruning problem as a non-convex optimization problem, solved it using the ADMM approach while adopting layerwise pruning, and demonstrated a maximum pruning of 71%. In contrast to these approaches, Deadwooding targets to achieve highly pruned networks (90% -99%) while maintaining accuracy as well as amplifying robustness.
Recent studies [39,17] established the existence of a randomly initialized sub-network in a larger model matching the benign accuracy of a pre-trained network of similar complexity. Additionally, Ramanujan et al. [39] demonstrated that highly robust sub-networks exist within complex networks. SoA approaches Sehwag et al. [45,47] aimed to compute robust pruned networks using a layer-wise pruning approach to remove an equal percentage of parameters from each layer, achieving a maximum of 90% and 99% pruning, respectively. However, recent works have demonstrated that for the same pruning amount, pruned models obtained using global pruning have better prediction accuracy than the models pruned layer-wise [1]. Some other works focused on different aspects of compression, such as quantization [37], Low-Rank Factorization [48], distillation [43], etc. In-time sparsification is one of the emerging approaches which intend to train an already sparse network while constraining the number of parameters, resulting in a sparse network with a different configuration than the original model [28]. However, this paper presents a novel robust pruning-based model compression mechanism since it has the advantage of using already trained complex off-the-shelf models and has been widely used in literature [23]. Thus, our SoA constitutes only model pruning approaches.

D Importance of various components of Deadwooding
This section aims to draw attention to the significance of some components of Deadwooding which results in a substantial improvement over the state-of-the-art. Section D.1 discusses the role of benign accuracy during sparsification and how it helps to achieve global pruning. Section D.2 demonstrates that Deadwooding captures the importance of different layers of the model towards model inference. Finally, section D.3 is a continuation of section 5.2 and explains the benefits of using FGSM-looping (a variation of FGSM) as an adversarial loss component during fine-tuning.

D.1 Achieving Global Pruning through Benign Accuracy (Continuation of Section 6.2)
We now show evaluations that demonstrate the importance of using benign accuracy in the sparsification training step (Equation (4) in the main paper) and how it enables global pruning while maintaining network connectivity. We also discuss the impact of using global pruning instead of restricted layer-wise pruning on the model's performance. The results presented here are computed for a VGG-16 network trained for the CIFAR-10 dataset when it is pruned by 99% using various approaches. Figure 4a shows the amount of pruning incurred at each layer using two variations of Deadwooding corresponding to the presence and absence of the term benign accuracy (L) in the loss function used during sparsification training step. According to this comparison, the pruned model obtained without using L is disconnected and possesses no remaining connections (i.e., parameters) in layers 9 − 13. Also, the accuracy of the model, thus obtained, is just 10% (as shown in Section 6, Table  4). In contrast, our presented approach (i.e., including L in sparsification loss) retains parameters in all layers contributing to active input-to-output path, resulting in a highly accurate connected sub-network. Furthermore, Figure 4b shows the comparison of the pruning percentage at each layer of the pruned model obtained using Deadwooding vs. Hydra. Hydra performs layer-wise pruning; hence the same percentage of parameters are removed from each layer. In contrast, Deadwooding performs global pruning; as a result, each layer of the pruned model has a different percentage of parameters removed. Since not necessarily every layer is similarly contributing to the model's inference, Deadwooding's global pruning effectively captures the disproportionate layer importance and achieves pruned models which exhibit high benign accuracy and robustness. Notably, for VGG-16, the initial and the final layers undergo a lesser pruning amount than the middle layers.  L a y e r 1 L a y e r 2 L a y e r 3 L a y e r 4 L a y e r 5 L a y e r 6 L a y e r 7 L a y e r 8 L a y e r 9 L a y e r 1 0 L a y e r 1 1 L a y e r 1 2 L a y e r 1 3 L a y e r 1 4 L a y e r 1 5 L a y e r 1 6

D.2 Pruning parameters accoring to the importance of a layers
Pruning Amount L a y e r 1 L a y e r 2 L a y e r 3 L a y e r 4 L a y e r 5 L a y e r 6 L a y e r 7 L a y e r 8 L a y e r 9 L a y e r 1 0 L a y e r 1 1 L a y e r 1 2 L a y e r 1 3 L a y e r 1 4 L a y e r 1 5 L a y e r 1 6 Pruning Amount

Chart Title
Hydra Ours (b) Layer wise pruning amounts for Deadwooding and Hydra for pruning for VGG16 with CIFAR10 dataset at 99% pruning amount Figure 4: Importance of using benign accuracy as a loss component during sparsification training 1. For smaller pruning amounts (20-50%), the layers 13-15 get pruned in a significantly higher percentage than others. This is because they are fully connected linear layers and have more redundancy than the initial convolution layers that extract meaningful features for effective classification.
2. As the required pruning increases, the pruning amount of the convolution layers also increases with the depth of the layer. This is because the initial convolution layers extract low-level features required in the following layers for high-level features extraction. Hence, initial layers are pruned lesser than the inner convolution layers.
3. The last layer, which decides the label of the input, gets pruned the least among all the linear layers. L a y e r 1 L a y e r 2 L a y e r 3 L a y e r 4 L a y e r 5 L a y e r 6 L a y e r 7 L a y e r 8 L a y e r 9 L a y e r 1 0 L a y e r 1 1 L a y e r 1 2 L a y e r 1 3 L a y e r 1 4 L a y e r 1 5 L a y e r 1 6 Pruning Percentage 20% 50% 90% 95% 99%   perturbed samples generated by FGSM are restricted to ± perturbation for each feature which bounds the perturbed samples to be on the corners of an L ∞ ball of radius max around the sample. Although the direction of the gradient for a sample may change in every training epoch, which helps FGSM to generate different perturbations for the same benign sample, but for a d-dimensional feature space, the size of this search space for FGSM is 2 d . Many recent works discouraged the use of FGSM for adversarial training because of its restricted search space [30,9].
PGD, however, is a multi-step attack, which generates a small amount of perturbation in one iteration and computes the gradient again to change the direction of perturbation in each step while keeping the perturbed sample in L ∞ ball of radius max . Thus, PGD searches for the perturbed sample in whole of the L ∞ space. As a result, PGD requires more computation time than FGSM to generate one perturbed sample.
Madry et al. [30] suggests that for small adversarial perturbations in L ∞ norm, FGSM can produce adversarial samples which are similar to the ones produced by PGD. Motivated by that, FGSMlooping helps achieve the same effect as PGD by generating adversarial examples for multiple values of ranging from ∼ 0 to max ; thus, increasing the search space of FGSM. Therefore, with FGSM-looping, we get an additional benefit of varying magnitude in addition to varying the direction of the perturbation in every epoch. As a result, we are able to achieve similar results as compared to PGD (step size = 10) adversarial training, but in three times lesser computation time (for the current system configuration).

E Miscellaneous Results
This section discusses some miscellaneous results which support the results presented in the main paper. Section E.1 shows the era evaluations of the pruned models against stronger adversarial attacks.
E.1 Testing with Stronger attacks for same perturbation budget (Continuation of Section 6.2) This section aims to demonstrate the ability of the pruned model to defend against stronger attacks. Table 4 compares the results for era for stronger attacks on a 99% pruned VGG-16 model trained for CIFAR-10 obtained using Hydra and Deadwooding. The evaluations demonstrate that Deadwooding outperforms Hydra by 6.69%, 13.97% and 52.06% era for attacks AutoAttack [13], FAB [14] and Deepfool [35], respectively. Notably, the perturbed samples generated by DeepFool [35] are not bounded by a perturbation budget ( max ), and Deadwooding outperforms Hydra in this case as well.
The stronger versions of PGD attacks tend to find adversarial examples in a fine-grained manner by increasing the number of attack steps numSteps while keeping the maximum perturbation max same. The step size α is computed using α = 2.5 * max /numSteps which is in line with the state-of-the-art [29]. Table 5 shows results for era for five pruning amounts, which are 99%, 95%, 90%, 50% and 20%, for VGG16 on CIFAR-10 dataset. We show the results for four variations of the numSteps, which are 10, 50, and 100. The evaluation shows that even for stronger PGD attacks, we achieve a similar amount of era with an average standard deviation of 0.0062 for all the pruning amounts.

F Comparison with Other Pruning Techniques
This section demonstrates the effectiveness for pruning mechanism used in Deadwooding as compared to some other pruning paradigms presented in literature: training from scratch (Section F.1), multi-step pruning (Section F.2) and least weight magnitude based pruning (Section F.3).
F.1 Comparison with training from scratch (Continuation of Section 7) Leveraging Deadwooding, training a smaller network from scratch can be considered a more straightforward approach (than pruning) to achieve compressed models. However, several prior works [27,22] showed that training a pruned model from scratch does not result in an efficient model and that retaining parameters from the initial training phase results in a better solution.
To evaluate that, we applied Deadwooding on a randomly initialized VGG16 model as the base model instead of a pre-trained one. The evaluation was performed on the CIFAR-10 dataset at 99% pruning amount. According to the evaluation, the resulting compressed model fails to achieve acceptable accuracy (only 10% eba); hence demonstrates that training the compressed model from scratch is not effective, which is in line with the state-of-the-art [47,55].

F.2 Comparison with Multi-step pruning (Continuation of Section 7)
This section demonstrates that Deadwooding results in pruned models exhibiting similar eba and era for single-step and multi-step pruning (also known as Iterative Pruning [33]). In multi-step pruning, each iteration constitutes two steps: Pruning and Fine-tuning. We start with a low pruning amount and keep increasing the pruning amount in each iteration until the desired pruned amount (in the compressed model) is achieved. We compared the evaluation metrics of the 90% pruned VGG16 model on the CIFAR-10 dataset using the presented single-step pruning approach and the multi-step version of our approach. We observe that the evaluation metrics for both approaches are comparable, with a variation of 0.3 percent points in the eba and era. Thus, we conclude that the multi-step pruning and single-step pruning result in similar pruned models, which is in line with state-of-the-art [47].

F.3 Comparison with Least Weight Magnitude based pruning
The Least Weight Magnitude-based pruning is a gold-standard pruning step technique used by several prior studies [47,53,45]. To compare least weight magnitude-based pruning with the Deadwooding's model pruning step, we performed least weight magnitude-based pruning (instead of Deadwooding's sparsification step) followed by our modified adversarial knowledge distillation fine-tuning step on the VGG16 network (99% pruning ratio and on the CIFAR10 dataset). The pruned model was disconnected and achieved only a 10% eba, evidencing that our presented global pruning step is more effective than the LWM-based global pruning approach.