DPP node pruning: [1]} uses DPP to propose a novel node pruning method for feed forward neural network. They define information at node \(i\)  of layer \(l\)  as \(\mathbf {a}_i^l (=(a_{i1}^l,\ldots ,a_{in}^l))\) , where \(a_{ij}^l\)  is the activity of node \(i\)  of layer \(l\)  on \(j^{th}\)  input. Here \(\mathbf {a}_i^{l} = g\left( \mathbf {b}_i^l \right)\) , where \( \mathbf {b}_i^l = \sum _{i=1}^{n_{l-1}} w_{ij}^{l-1} \mathbf {a}_i^{l-1}\)  is the information at node \(i\)  of layer \(l\)  before activation. A layer is pruned by choosing a subset of hidden nodes using a DPP kernel: \(\mathbf {L}\)  (\( = \mathbf {L^{\prime }} +\epsilon \mathbf {I}\) ), where, \(\mathbf {L^{\prime }}_{st} = \exp ( -\beta \left\Vert  \mathbf {a}_s^l - \mathbf {a}_t^l \right\Vert ^2 )\)  and \(\beta \)  is a bandwidth parameter. The matrix \(\mathbf {L}\)  is of dimension \(n_l \times n_l\) , as total number of nodes in layer \(l\)  is \(n_l\) . By the property of DPP, this procedure will keep a diverse subset of nodes for each layer w.r.t. information obtained from the training data. A reweighting technique (see Section 2.2 of [1]}) is then applied to outgoing edges of retained nodes to compensate for information lost in that layer due to node removal. Note that DIVNET denotes DPP node pruning with reweighting as in [1]}.
<TABLE>